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ABSTRACT 



This document contains a final report, evaluation report, 
and materials from the Better Educational Skills Training (BEST) workplace 
literacy program in Vermont. The seven-page final report describes program 
components: (1) expanding and institutionalizing the workplace literacy 

program developed through two previous national workplace literacy 
demonstration grants; (2) documenting and disseminating its promising 
practices; and (3) delivering workplace education in three Vermont counties 
to employees of firms in manufacturing, construction, health care, and small 
business sectors. A total of 1,081 employees of 12 employers were served 
during the project. Attached to the final report are a review and articles 
about project publications. The external evaluation by Donald J. Leu 
concludes that the project met its major goals for improving workers’ 
literacy levels and contributing to positive changes in the organizations. 
The evaluation's appendix describes the evaluation plan, provides forms used 
to collect formative and summative data, and addresses the process decisions 
involved in the use of the evaluation instruments. The last section of the 
document presents results of program evaluation interviews conducted with 
employees at participating companies by Robert T. McLaughlin. The following 
conclusions are offered: respondents agree that the quality of instruction 
was high; students and instructors appeared willing to adapt to each 
workplace's logistical constraints; and respondents wanted more employees to 
benefit from the program, but felt that some stigma was associated with 
participation. (KC) 
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® Points of view or opinions stated in this 
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official OERI position or policy. 



Program Description 



The Vermont Institute for Self-Reliance, d/b/a Vermont Adult Learning 
received this Demonstration Program grant to 

1) expand and institutionalize the Better Educational Skills Training (BEST) 
workplace literacy program developed through its two previous national 
workplace literacy demonstration grants; 2) document and disseminate its 
promising practices; and 3) deliver workplace education in three Vermont 
counties to employees of firms in manufacturing, construction, health care, and 
small business sectors. 

This program is unique in its development of comprehensive strategies for 
responding to the context and culture of each workplace in its mode of service 
delivery and the content of its courses and workshops. This responsiveness 
stems from our commitment to an employee-led process of identifying 
workforce educational needs and interests and developing a long term plan for 
addressing them. The BEST program was able to custom-design curricula to 
meet employees’ needs and demands while consistently achieving quality, 
because the foundation for all instruction is a problem solving and "learning to 
learn philosophy linked to the SCANS. skills. Thereby, BEST builds employee 
self-confidence, flexibility, and promotability, while strengthening attitudes and 
skills necessary for lifelong learning. 



The educational service delivery strategy includes individual learning plans, 
individualized instruction, small instructor-led classes, topical workshops, 
cooperative learning, alternative assessments, and a unique interactive 
instructional software tool called "Responsive Text." A total of 1,081 individuals 
working for 12 different employers were served, as the table below shows. 



Employer 

Rutland Regional Medical Center 
General Electric Aircraft Engines 
Northeast Cooperatives 
Vermont Store Fixture 
Fleming Oil 

Southern Vermont Home Health 
Ideal Horizons 
Geka Brush 
Middlebury College 
Carris Reels 

Bread Loaf Construction 
Russell Corporation 
Total employees serve 



# served 


Sector 


339 


health care 


170 


manufacturing 


65 


wholesale 


7 


small business 


83 


small business 


76 


health care 


49 


small business 


35 


manufacturing 


123 


custodial 


127 


manufacturing 


6 


construction 


1 


constuction 


1,081 


unduplicated count 
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Fiscal Data 

The project period initially was October 1, 1994 to September 30, 1997. The 
project period was formally extended to February 2, 1998 in order to complete the 
dissemination of project products and allow the project director to be a presenter 
at the 1998 Winter Institute on Adult Basic Education in the Workplace jointly 
sponsored by Georgia’s Life Long Learning Network and The Centre for Literacy 
in Montreal. The grant award for the three year (plus four month) project 
totalled $776,854. A recipient cost share (match) of $343,042 was required. The 
total match (in kind and cash) exceeded $420,988. Of this amount, $133,265 took 
the form of cash contributions from the business partners towards the cost of 
classes for their employees. 



Budget Period 

10/1/94-9/30/95 

10/1/95-9/30/96 

10/1/96-2/2/98 

Total 



Fed $$ awarded 

$276,236 

$293,549 

$207,069 

$776,854 



MATCH 

In-Kind 



OBLIGATED 

Total Match 



$59,964 

$152,929 

$74,830 

$287,723 



Cash 
$0.00 
$20,822 
$112,443 
$133,265 



$59,964 

$173,751 

$187,273 

$420,988 



Some Successes & Challenges 
Products 

One lasting success is our dissemination products. The strategy of full-time adult 
educators writing for their peers set our products apart from others. Our 
products are proving valuable to adult educators in this country and abroad as 
. well as being useful in Vermont Adult Learning's welfare to work programs. 



Evaluation Design 

Our triangulated evaluation design was quite successful at capturing impact at 
multiple levels: customer satisfaction, learning gains, and job performance. Our 
external evaluator worked closely with the teaching staff to design the 
evaluation procedures. Our instrument for assessing strategic knowledge was 
tested and revised repeatedly to create an open-ended yet scorable format that 
allows learners to freely express their experiences and documents the effects of 
our teacher’s emphasis on metacognition. We also collected success stories to 
provide qualitative examples of the gains measured. A selection of these are 
included in the appendix. 

Learner's Lives 

The impact of the program on individual learners lives is most certainly of 
lasting value beyond the end of the project. Skill gains were accompanied by 
increased confidence, self-esteem, and belief in oneself as a learner. In a number 
of cases this resulted in dramatic attitude and job performance changes. Teacher 
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-p. -MstarHandly. describes one such case in this account of a long-term reading 
student of hers at a manufacturing plant. 

"Imagine sitting in a business meeting and being unable to 
v - : : : - is saying: One of my 

students, John, experienced almost continual rage at situations like this. 

. He. was a very hostile young man, with not just a chip but an anvil on his 
shoulder. In other areas of his life, he had successfully isolated himself 
from situations where his inadequacy could potentially prove 
embarrassing. The work place was different, however. There he was 
stuck. He had to participate in team meetings. When John asked me why 
the speaker couldn't have found simpler ways of expressing himself, 
rather than using so many "big" words, I had to explain why people used 
comprehensive terms, rather than a entire sentence, to define a concept. 
When we considered how large an instruction manual would have to be 
if single words were eliminated and replaced with 10-15 word sentences, 
he got the picture. But he was still ticked off. Anxious to help him with 
his difficulties, and realizing that despite his progress whith his reading, it 
would be years before he would be able to enjoy adult-level printed 
materials, I suggested he get a library card-that was a first!-and check out 
- - some audio books. _ 

Still hostile, John trotted off to the library. He listened to books in 
his truck going to and from work. He listened at his workbench. He 
listened doing chores. At first he kept phonetic lists of words he didn't 
understand so we could discuss them in class. As his comprehension 
skills using context clues increased, so did his confidence. He read more. 
And more. The man who never had a library card in his life now reads 5- 
7 AUDIO BOOKS A WEEK. 

Slowly John came out of his isolation. His intellectural curiosity 
grew. He got braver. He began attending movies made from books he'd 

read. He even attended a play with a supportive neighbor, whom he 

could occasionally poke to ask a question. John's hostility began to 
evaporate as his understanding of his world grew. 

His manager wrote me a wonderful note on the annual survey we 
take of employees' on-the-job changes since participating in our program. 
He said John was a "changed human." Instead of being continually angry, 
he had become cheerful and helpful on the job, speaking up in team 
meetings and even volunteering for new assignments rather than 
complaining about what he was given to do. We both noticed a new 
spring in John's step. 

And the anvil was gone." 
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Reaching Non-Readers 

In Vermont isolation, embarrassment, and fear are major barriers to progr am 
irtVoiv^ment for^^m who see themselves as non-readers. To overcome 

this difficulty we attempted to briefly and informally interview every employee 
in each Of the workplaces." We offered individual tutoring to those who weren't 
ready to take a class. We also presented the B.E.S.T. program as a brush-up 
program for everyone. Still we know from supervisors and co-workers that 
there were individuals with substantial reading difficulties who never 
volunteered for the program. 

Sustained Business Involvement 

Aligning the B.E.S.T. program with the firms' business goals was elusive. We 
pursued a number of avenues toward this end. We established planning and 
evaluation teams at each business with employees, supervisors and 
management represented. We brought the business partners together with our 
external evaluator and our teachers to explore criteria for success and how the 
program could contribute to the bottom line. We made presentations to 
management committees. We sought a strong investment in the program from 
all stakeholders in each business; however, support was uneven. Most of our 
partners did continue the program-and pay for it~for most of the year following 
the end of the grant. However, as of January 1999 we will be teaching only a few 
classes at a few businesses. Several employers ended the program because of 
budget cuts or business down turns. Others felt the program had run its course 
and was no longer enjoying strong enthusiasm from employees. 

Marketing 

Our plan to use this, our third workplace literacy demonstration grant, to 
institutionalize workplace education was on target. The strategy of asking 
business partners to contribute toward the costs of classes in years 1 and 2, and to 
make a cash contribution to cover the cost of the teacher in year 3 was effective. 

It forced us, as a nonprofit accustomed to providing "free" services to think in 
business terms. The employers shifted their thinking and expectations too, as 
they paid more for services. Thus we had a year of grant funding to adjust. 
Unfortunately, we had not anticipated the cost or the time involved in 
developing a business and marketing plan, producing marketing materials, and 
marketing the program. Had we been able to complete all curricula products in 
year 2, we might have been reasonably successful in institutionalizing and 
marketing the program in year three. All the same, Vermont Adult Learning 
did commit organizational resources beyond the grant to marketing the program 
and has embraced workplace education as one of its services. We did attempt to 
aggressively market workplace education beyond the end of the grant, but budget 
constraints have forced us to abandon that position. Still we do publicize the 
availability of workplace education and respond enthusiasticly to business 
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;v: inquirees.-^ Although small, workplace education is continuing as a service of 
Vermont Adult Learning. 

Outcome Goals Achelved 

Outcome goals 1, 2, and 5 were addressed by the external evaluation conducted by 
Dr. Don Leu. These goals— all of which were acheived— are reproduced below, 
and the Executive Summary of Dr. Leu’s evaluation is reproduced on page 7. A 
detailed discussion can be found in Dr. Leu’s complete Final Report (dated Feb. 8, 
1998) in the appendix. Please note that permission was granted on 9/20/95 to 
modify outcome #5 and its objectives to the statements printed below. 

1. Increase the literacy levels of targeted individuals especially in 
regards to reading, writing, math, communication and problem solving 
tasks in the workplace. 

2. Demonstrate that completion of the BEST program results in 
positive organizational outcomes such as improved employee 
performance, especially increased involvement in activiites designed 
for continuous quality improvement. 

5. Dr. Leu will conduct a systematic independent formative and 
summative evaluation of the project. 

5.1 Dr. Leu will design an evaluation that is grounded in the instructional 
goals of the project and the concerns of the business partners. 

5.2 Dr. Leu will conduct an evaluation of student learning gains and the 
effects on job advancement and job performance. He will also evaluate 
project impact on the businesses. 

5.3 (deleted). 

Outcome goal #3 was revised during grant award negotiations as follows. 

3. Assess the impact of BEST on the social dynamics and culture of 
the workplaces and on productivity, job retention and career 
advancement. 

Dr. McLaughlin interviewed a range of stakeholders focusing on their 
operational definitions of effectiveness. The interviews with education 
department and human resource leaders in particular revealed impact on social 
dynamics and culture. Dr. McLaughlin’s report (May 1995) is in the appendix. 

4. Create curriculum and staff development products and disseminate 
project results widely. 

We proceeded according to our federally approved dissemination plan and 
approved revisions to it. We mailed copies of all products to each State's Literacy 
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- Resource Center and each NWLP project director as well as national 
clearinghouses and ERIC. We also posted information to the NIFL-Workplace 
Literacy listserv and mailed information to State Directors of Adult Basic 
Education. We have presented workshops and displayed products at major 
national conferences ^^induding- AAACE^GOABE^ Workplace Learning 
(Milwaukee), and the Winter Institute (Atlanta). The paper presented in Atlanta 
by Judith Lashof and Mike Hillinger is in the appendix. Selections from our 
book Reflections on Workplace Education are on-line with the Hub 1 web site 
(formerly. Eastern LINCS) and Mike Hillinger of Lexicon Systems has established 
a web site (www.lexiconsys.com) where visitors can actually use Responsive Text 
Software. - 

We've just sold three hundred copies of our Communication for Workers 
curriculum to The Center for Literacy Studies at the University of Tennessee for 
distribution to Adult Basic Education teachers in that state's welfare to work 
programs. We are in dialog with New Reader’s Press about distributing 1 
Workshops for Workers and Communication for Workers . Presently, copies of 
all of our dissemination products can be purchased directly from Vermont Adult 
Learning. 

All of our print products were created by teachers for teachers. Teachers played a 
major role in the development of the video and the software. Our product list is 
in the box below; an order form is in the appendix. 



Curriculum and Staff Development Products 

Applied Learning on the Oregon Trail 

Communication for Workers 

Confidence in the Workplace (14 minute video) 

Reflections on Workplace Education: Teachers Talking to Teachers 
Responsive Text Software Macintosh Demo Disk and Manual 
Communications Skills 

Hazardous Communication & Tools for Quality 
Workshops for Workers 
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Executive Summary 



P resents report for the BEST project and evaluates the extent to which 

the BEST Program met its initial objectives. In general, we have followed the evaluation plan 
. . . . ... . •• :*..*.deveIoped.vin.;"the '.*firsr- *year;. of this project. Instruments and -procedures are included in the 
Appendix. Summative data are reported here; formative data have been reported on an on-going 

The major findings from the evaluation include: 



• Statistically significant gains appeared on the strategy inventories for participants who worked 

. _ . on reading strategies, communication/problem solving strategies, and writing strategies but not 

for participants who participated on math strategies. Both teachers and participants reported 
seeing gains in reading, writing, math, communication and problem solving strategies for 
participants who worked in these areas. Supervisors/co-workers saw gains in each area with 
the exception of math strategies. 

• Statistically significant gains appeared among participants working at the two highest levels of 
reading comprehension but not for those working on level one. Among those working at level 
one, statistically significant gains appeared in decoding scores. Gains in reading ability were 

perceived by participants, teachers, and supervisors/co-workers. 

• Statistically significant gains appeared among participants working on improving their writing 

workers 1 Wntmg ablllty were also b y participants, teachers, and supervisors/co- 

’ * Statistically significant gains appeared among participants working on improving their math 
Gams m math ability were also perceived by participants and teachers, but not bv 
supervisors/co-workers. J 

• Statistically significant gains appeared among participants working on improving their work- 
related communication skills and problem solving. Gains in work-related communication skills 
workers em S ° Vmg were P erceived b y participants and teachers, but not by supervisors/co- 

• P e . rceived ? e y had made “good” progress on their top three learning goals during 
both the first six-month and the second six-month period. 

• Participants and supervisors perceived increases in a wide range of areas associated with 
positive organizational outcomes. The greatest increase appears to have occurred in 
participants interest in continuing their education. 

In summary the BEST project has met the major goals established at the beginning for improving 
literacy levels (reading, writing, math, communication and problem solving) and demonstrating 
hat participation in the BEST program results in positive organizational outcomes. Many lives 
have been fundamentally improved by the results of this program; many positive changes have 
taken place within the organizations in which participants work. 



Prepared by Donald J. Leu, Ph.D 
Professor of Education 
Syracuse University 
203 Huntington Hall 
' Syracuse. NY 13244 

315-443-4757 
djleu@syr.edu 




Rutland, Vermont 
February 8, 1998 
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use, professional, attractivefonnat-lookno further! 



in an easy-to- 



. These workshops were created by teachers for ieachers' They're learner-centered, highly interactive, 
and truly teacher-friendly. - Activity-based, the teaching guides are simple for both teachers new to 
workplace education and experienced teachers to use and adapt to their specific circumstances. 



Workshops for Workers — 3-hole punched, shrink-wrapped and ready for 
insertion into your own binder: \ 

5:-; Business Writing: Simple and Strong (6 hrs)*Better Writing on the Word 

Processor (6 hrs.)*The Challenge of Change (10 hrs.)*Communication on the 
■ ' Telephone (2-3 hrs.)»Growing in Self-Confidence (10 hrs.)«lntroduction to 



(7.5 hrs.)*Funny, 
Group (5 hrs.) / 



.^•g?$.;;:8^ ? d0gg- -v Qt v c Total. 



in a 



^Applied Learning on the Oregon Trail ^stapled -- how to use MECC’s Oreaon 
Trail II . 



y:y \ V- 



: collecting and analyzing 
resources, working in 



information, 

• teams, 

j and desig ning systems, as wel I as learni ng o n^demand. 



Communication ^for Workers - activity-based; teach i ng /guide, - spiral bound: 



People (2 hrs.)«Communicating in Groups (2-3 hrs.)*Working in Teams(2-4 hrs.) 



y ’$^$^20.6b^g^ Qt v - ^/-T otal 

v Rocnnnciuo Tovf YMoa nicl/\ ‘ A m m i m a O !#• II a 4 w . aI — mm a O 



Responsive Text (Mac Disk) ~ “Communicatibn Skills 1 " demo & manual -- 
. •: provides a powerful way to develop complex reading skills with students in control of 

•/' ;iS/\ theprocess.'^ W'-'r' / ; -';d$-25.00^^-' QtyV^5-Total 






Video: Confidence in the Workplace '-yi 4 mins. students, 
teachers, and employers explore impact of education Droaram on the workplace. 






Complete Sef (SAVE $24!!) 



$35.00 
$1 50.00 W. 



Qty 

Qty 



Total 



;>J;f^j)dstage @ 9.5% & tax (ifapplic., or exempt #) 
...... ? plus handling - -V/' -'V’:- 

GRAND . TOTAL%^ : r : :^ ■: 

yShip Jo: ■ • ' • 



Total 



4.75 



. Thank you for your order! 



ERIC Central Office: RR*1, Box 283 BB / East Montpelier / Vermont 05651 / 802-229-5221 / Fax: 802-229-5238 



Review of "Reflections on Workplace 
Education: Teachers Talking to Teachers" 






' ■' E very educator I know has Come to appreciate 
and even rely on the growing number of 
homegrown publications that teachers are writing 
- — r - ---.for teachers. The -teachers in Vermont’s BEST 
program have recently added a new contribution 
to this collection, a description of their workplace 
education program called Reflection on Workplace 
Education: Teacher s Talking to Teachers . This 
text is accessibly structured around the “essential 
questions” the staff had to answer for themselves 
as they implemented workplace programs in a 
variety of businesses. Their questions lead us 
through the areas of program design, teaching 
.. ... ...philosophy, curriculum development, and 
workplace education issues. 

As a workplace educator, I came to the text 
wondering if it would address the problematic 
issues that I face time and time again (that seems 
to be a major role for peer publications - we turn 
to them, for assurance that we are not uniquely 
“challenged”.) And it does, with recognizable 
anecdotes about running programs in settings 
where you have many players to satisfy but limited 
control. 

Having found my connection, I went on to 

read about the key elements of BEST’s approach. 

T.he program is committed to student-centered 
learning, involving workers from the beginning 
in employee education committees that survey 
workers about their educational needs and 
interests. Classes emphasize metacognition, 
teaching people to notice how they think and learn, 
and evaluation is done collaboratively rather than 
through standardized tests. 

One missing piece that would have been 
helpful is more background on who the students 
are and what these particular workplaces are like. 
These students seem more educationally advanced 
than most of the students we work with in the 



Massachusetts Workplace Literacy Consortium. 
Still, we share many of the same dilemmas - worker/ 
students who fear that participating (or not 
participating) in classes will target them for lay-off, 
supervisors who are reluctant to release workers for 
class, etc. - as well as similar rewards - students who 
take this sometimes difficult step for themselves, 
workers who gain confidence and become more 
active in and out of work, and employers who begin 
to recognize the strengths of certain employees. 

“Reflections” offers us a glimpse into an area of 
adult education that many teachers don’t have access 
to, as these programs are delivered at worksites and 
away from learning centers where teachers’ paths 
cross. If you’ve been wondering what happens in 
such programs, the BEST teachers offer one place 
to begin. 

Reflections on Workplace Educatio n: Teachers Talking, to 
Teachers was written by Pamela. Letoumeau-Fallon, Kate 
Taylor, Laura Chase, Candice Kramer, Judith Lashof, Judy 
Palmer, Dorothy Ziegler, and students, from the Better 
Education Skills Training Program (B.E.S. T.) of Vermont 
Adult Learning. For more information contact the 
B.E.S.T. program at (802) 775-0617. 



Abouf NELRC ... 

The mission of the N e w : Erigland Literacy ; 
; . Resource Center is to foster cdUaboratiom ^ |f§| 
i stiiflriii'g iamongi: the.six.New^glah.d : statttih.v:: 
literacy-related professional development^ < 
program innovation, inlormati<m;r^urc« irid 
technologies in order: to enhan^ithAd^ 
basic educational services to adulj^r^|^|^EC6 
News is a supplement to each New J&gl^ 

Newsletter, and is edited by Siija K^enl^ch, 

Coordinator of the NELRC. A 
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(Dr ^ichddHillmger is President of Lexicon Systems in Sharon, Vermont. ■ He carl be reached via e-mail'at mlh@lexiconsys.com. 
Judith Lashofis Director ofJJrants at Vermont Adult Learning in Rutland . She can be reached via e-mail at JudyLashof@aoiconu) 



.In i990 I^xfcon*Systemsibegan \vbr 
Adult Learning on a federally-funded workplace literacy 
project.* Lexicon explored how the emerging multimedia 
Capabilities of computers could be used to assist workers 
in reading training manuals, learning safety procedures, 
and other job-relevant tasks. The result is an approach 
called Responsive Text™. ;/ 



Responsive Text and Reading Theory 
Responsive Text is based on what we know about fluent 
readers. Before describing how the program works it is 
useful to review what research has found about the 
reading process and how these findings are reflected in 
Responsive Text, j 

Reading as a multi-level process 
Most theories of reading describe comprehension as a j 
multi-level process. and proficient readers bring many 
skills to the reading task. These include: 
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Metacognitive knowledge" 



Discourse knowledge 



Syntactic knowledge 



Vocabulary knowledge" 



Decoding knowledge" 
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Figure 1 . A multilevel model of reading 
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Deco ding Knowledge . Atthe most basic level.-reading 
proficiency requires decoding visual representations of 
words into a phonological representation (Gough & 
Hijlinger, 1980). While poor readers may be able to 
decode words, the process is often difficult and diverts 
attention from higherjlevel comprehension. 

Ypcabulary knowledge. As Chall (19^83) notes, reading 
beyond the earliest stages requires a store of background 
knowledge. Knowledge of vocabulary, phrases, and 
idioms, all gained through reading, is necessary for later 
reading stages. Poor readers are often deficient in the 
necessary background knowledge to understand a pas- 
sage. 



S yntactic knowledg e All languages follow predictable 
rules of grammar. While most of us cannot state these 
rules, we implicitly use them to form grammatical spoken 
sentences. This knowledge of syntax provides clues that 
a good reader will use in comprehending written language. 



Discourse knowledge . Written language has itsown 
structure that a good reader can utilize. Narratives 
generally follow a common form of introducing charac- 
ters, plot, and resolution. ■ As readers gain Experience 
they can use this “story grammar” to guide their reading. 
Expository text also has many structural cues, such as 
text headings and subheadings, that provide valuable 
information. 

Metacognitive knowledge . Good readers monitor their 
comprehension, continuously asking “Does this make 
sense? What am 'i doing? Does it work?” This mqnitor- 
ing allows the readers to adjust their reading to meet 
different task demands. Baker & Brown (1984) have 
noted that poor readers often have difficulty monitoring 
their comprehension of a passage. 

Non-fluent reading to automaticity 
How and when these processes- are used depends on-the 
’ v fluency of the reader and the difficulty of the reading 
material. At the earliest stages of reading development 
the reader’s attention is focused on-decoding and under- 
standing simple- sentences. With practice' these lower- 
level processes become more automatic, allowing the 
reader to shift more attention to higher-level processes. 

N f 

Mature readers can shift the relative importance of these 
processes with the difficulty of the text. With famiTiar 
material, fluent readers focus their attention on higher- 
level comprehension; but when confronted with unfamil- 
iar and/or more complex material, they may shift their 
focus to lower levels. *•. 

Responsive Text design 

This theoretical framework lead? to these design princi- 
pals: ' 

Use multimedia to support reading comprehension . Poor 
readers may need- help at one or more levels of reading, 
and Responsive Text uses the computer to provide this 
assistance (Figure 2). A reader who has difficulty 
decoding can have the computer provide a spoken 
version. When vocabulary is weak, the system can 
provide background of varying complexity. 

Encourage reading as an active process . Higher-level 
comprehension requires that readers ask questions, make 
predictions, and find information. To encourage this. 
Responsive Text poses questions, provides writing 

(continued on page 19) 
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opportunities, and allows the reader to explore more 
detailed information. 
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Figure 2. Responsive Text Support Structure 

Integrate multiple forms of literacy . As Kirsch & 

. Jungeblut (1986) have demonstrated, literacy requires 
more than knowing how to read; what they refer to as 
Prose Literacy. In real life we have to read and interpret 
charts, tables, and graphs, which thev call Document 
Literacy. We also must frequently araw upon math skills, 
which they refer to as Quantitative Literacy; to compre- 
hend numerical relationships and evaluate simple data. 
Responsive text supports skills in prose, document, and 
. quantitative literacy, - 



View literacy contextually . Literacy should be viewed 
within the context of the reading task. While basic skills 
instruction is often considered as a prerequisite to training 
job-relevant skills, the distinction between a basic skill 
and job skill is not always clear. Responsive Text 
embeds basic-skill support and instruction within job- 
relevant content. If a reader needs help on a graph in the 
Basic Statistics component of Tools for Quality or a new 
medical term in Bloodbome Pathogens, Responsive Text 
can provide the basic skill instruction within a relevant 
context. 

By making basic skill support an integral part of the 
lesson, Responsive Text removes the distinction between 
training basic skills and job skills. It provides a bridge 
between programs that provide extensive but generic 
basic skill instruction and commercial job training 
programs (See also Hillinger, 1992). 

Responsive Text Overview 

Moving through Responsive Text is similar to turning 
pages in a book. To move to the next ‘'page/' click on 
the arrow in the upper right. To move to the previous 
page, use the arrow in the upper left. While you could go 
through all of the lesson using only these forward and 
back arrows, it is more likely you will use some of the 
supports accessed through the other screen buttons. 

These supports include: 

Speech 

Decoding words — the action of transforming written 
symbols into sound using rules of pronunciation — is 
frequently the first stumbling block encountered by poor 
readers. Responsive Text provides digitally encoded 




[shov background vord; 
frhov cltac op Icon 

Figure 3. Responsive Text Screen Layout 

speech to assist decoding. When the speech function is 
ON, speaker buttons are shown to the left of each 
sentence in the text. Click the speaker to hear each 
sentence read. While the sentence is being read it is 
highlighted in blue. 

Background 

Even with adequate decoding skill, comprehension of 
more complex material is difficult without sufficient 
background knowledge. When background is ON some 
of the words in the text are underlined to indicate that 
more information is available. Click on these underlined 
words and its definition is dis- 
played in a 
small win- 
dow. For 
some 
words, the 
dictionary 

also contains a more 
about... button that 
leads to even more 
information about 
the topic. 







I display more info 

[ (net aVays ayalablej 



dose window 



Closeups 

Even with decoding and background support, under- 
standing text, diagrams, graphs, and other information 
can be difficult. Closeups provide general purpose 
support for anything on the screen that may need more 
explanation. The explanation could include a page 
summary, a pictorial representation, or even an anima- 
tion. When you turn on Closeups, the areas of the screen 
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that have closeups are designated with a small magnify- . 

' ing glass icon. 'Clicking each' icdhreveals the additional 
'■•T-T information. -••••-- : : 

. , V'v . ■ 

Check up Question 

Reading is an active process, and Responsive Text uses 
Check up questipns’to insure you understand what~ 
you’ve read-thus far. Check up questions also encourage 
.. comprehension monitoring — the active checking of 
* r ^understanding while reading. Check upslrange from 

simple triie-and-false questions to engaging simulations. 

To encourage you to tiy the checkups, no performance 
record is kept. .Figure 4 shows an example of a check up 




Figure d. Check up question on inferences 



Student Pack— Writfhgand annotating 
Writing can beir useful S^ay to furtfier explore what you 
, are reading. A key component of the most recent 
version of Responsive Text is the capability to write, 
communicate, and annotate. These features are built into 
a.suitoof tools called the, Student Pack. 

The Student Pack is a 10-page notepad that can be 
opened at any time. In addition to keeping notes you 
may also send these notes as e-mail to others registered 
on the computer. , 

You can'also place, text as a “sticky note” on any Re- 
sponsive Text page. Sticky notes can be comments, . 
suggestions, or anything that you want to say. They 
offer the chance to “talk back.” 

To encourage writing, some pages have an open-ended 
writing suggestion available from the Write About 
button. 
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Implementing Responsive Text, the Practicai Side 
Selecting Content - • 

Responsive Text was developed as a component of each 
of Vermont Adult Learning’s three Workplace Literacy 
- Program FederalDemonstration grants'. In the first 
grant, the employer’s training manuals were converted to 
Responsive Text. Ih theory, this was a wonderful 
design for teaching the comprehension strategies, 
vocabulary, and knowledge needed for literacy in the 
workplace. However, reality did not comply. Shortly 
• ’ after starting to use Responsive Text in our literacy 
classrooms, the pompany Suspended its training classes 
for allworkers in favor of a more hands-on certification 
process. In addition, a number Of workers in our classes 
reported that they wanted to read something more \ 
interesting than company training manuals. ^ 

Iri the second gfartt we emphasized more generic training 
topics including OSHArequired training.^This content 
did not change rapidly and the topics were relevantto 
many companies. Blood Borne. Pathogens and'Hazcom 
were ^particularly well received. We developed Blood . 
Borne Pathogens with the training-department of phe ■ 

' Rutland Regional Medical Center and provided it to new 
housekeepers, guest services,' and nursing assistants. 
Several small companies without in-house training' , 

. departments contracted with us to provide the required 
Hazcom training for all their employees. Nearly all 
employees reported that they learned the information ~ 
fully and enjoyedjthe features of Responsive Text. 

Those who had experienced .lecture or video tape' 
Hazcom “training, reported that they learned far more 
from ResponsiveText. 'In addition, delivering tlys 
training allowed us toinformally observe' worker’s 
literacy skills arid refer those workers who needed all of 
the Responsive Text supports to Brush-Up classes. . - 

We expanded our range of topics with our third grant to 
include Assertiveness, Sexual Harassment, and a GED 
oriented unit in the form of a newspaper. Our intention 
was to generate greater learner ana teacher interest in the 
subject matter. However, these topics were delivered on 
a CD-ROM and we were able to purchase only one CD 
ROM computer for each ofour work site classrooms. 
Although the “Bulletin Board” and “Sticky Notes” 
features of this enhanced version of Responsive Text 
support creative use of the software in our computer 
classrooms, our teachers rarely organized their classes to 
take advantage of this. 

Teachers. Technology, and Reading 
Responsive Text is a rich and sophisticated learning tool. 
For teachers to embrace it arid use it to its fullest extent 
the following conditions should be present: 

1. Easy access to Responsive Text 

2. Responsive Text valued as a powerful learning 
tool 

3. Teaching methodologies that encompass com- 
puter-based learning as integral to instruction. 

(continued on page 21) 
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[ to. use 

computers in the classroom.' Our teachers were faced 
with installing Responsive Text on several different 
computer configurations and only one or two computers 
per classroom were capable of running the newest 
version of ResponsiveTexL Ideally, teachers new to 
technology would have classrooms equipped with the 
same model computer for every one or two students and 
someone to call on to manage the computer system and 



Easy Access. Vermont Adult Learning's use'of Respon- to each learner will enhance metacognitive awareness. 



In a reading and writing brush-up class. Responsive, Text 
cambe-usea in a wide variety of ways. The writing and 
message tools can be used to facilitate written dialog and 
an “informed conversation” that develops naturally. 
Alternatively, after reading a selection through once, 
learners can be directed to reread it to complete an 
assignment focusing on a feature such as vocabulary that 
is the topic of the class. The Responsive Text, lesson can 
be selected for the informational content and can be 
integrated into a thematic unit such as Sexual Harass- 
ment. Alternatively, the lesson can be selected to focus 
on a type of text, such as documents and-the variety of 
tables and charts that are embedded in them. 

A variety of classroom management strategies are 
possible. The teacher can work with half the class, or a 
small group of students while the others use the comput- 
'ers. If there are sufficient computers all students can use 
them at the same time. Once.students are able to use 
Responsive Text independently, the teacher can hold 
short individual conferences while the others are work- 
ing:. 

ummai 



install Responsive Text for easy access by students and 
Alternatively, it could be used m a computer 

computer- 



teachers. 

lab bv teachers experienced in managing 
based learning centers. 

Valuing Responsive Text. Responsive Text is a powerful 
way to develop Complex reading skills and strategies 
with the student in control of the process. Teachers who 
view reading as a simple linear process of decoding 
followed by identifying main ideas and details are likely 
to view Responsive Text as too complicated for their 
students. ' Teachers who have internalized a multi- 
dimensional model of the reading process will under- 
stand that Responsive Text mirrors the complex nature 
of reading itself: :;They will appreciate the control that it 
gives each learner to ^select and use multiple reading 
strategies and the instructional power of making these 
strategies and their selection visible and conscious. 

Teaching Methodologies. Teachers who rely primarily 
on lecture and group discussion or on drill and practice 
will find little place for Responsive Textin their classr . 
rooms. However, Responsive Text can be effectively 
integrated into a wide variety of learning environments. 

For example, in an individualizedjeaming lab environ- 
ment, Responsive Text can be used for self-paced 
instruction. In this environment the teacher will need to 
take care to introduce each feature of Responsive Text to 
learners and follow-up over several sessions to insure' ~" 
that learners are familiar with-WHAT each feature can 
offer, HOW to access each feature, and WHEN in their 
individual reading process tp do that. A periodic discus- 
sion with a small group of learners reflecting on the 
Responsive Text features and reading strategies helpful 



Text can be used flexibly in a wide variety 
of learning environments and for a range of purposes. 
TTie power of Responsive Text can be unleashed by a 
capable teacher wno appreciates the complex nature of 
reading, creatively structures learning experiences, and 
has adequate access to technology resources. 
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Success Stories in Literacy, Learning, and Workplace Performance 
— -v B.E.S.T. 

Purpose: The purpose of this form is to capture a brief description of a successful episode 

about how this project has increased literacy, learning, and workplace performance. 
vv -.= TeaeherSy ; employee participants, supervisors, or 'employer liaisons are all 
encouraged to complete this form in order to help us document these success 
...... stories. . . - 

Directions: Please fill in the date, location, author, and topic information. Then describe the 
success story as clearly as you can. Tell us what happened and why you think this 
. Js a success story. Use quotes where possible so we can hear the voices of 
participants. Use only first names. You should also use pseudonyms for 
individual and company names if you believe that the information you provide 
might offend anyone if they were to read it. 
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May we share this story with others? Yes 
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Feel free to continue on the back of this page, if necessary. 
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Executive Summary 



This evaluation presents the final report for the BEST project and evaluates the extent to which 

objectives.. . In general, we have, followed the evaluation plan 
developed in the first year of this project. Instruments and procedures are included in the 
Appendix. Summative data are reported here; formative data have been reported on an on-going 
. basis. : • * * 

The major findings from the evaluation include: 

- ... Statistically significant gains appeared on the strategy inventories for participants who worked 

- - - on reading strategies, communication/problem solving strategies, and writing strategies but not 

for participants who participated on math strategies. Both teachers and participants reported 
seeing gains in reading, writing, math, communication and problem solving strategies for 
participants who worked in these areas. Supervisors/co- workers saw gains in each area with 
the exception of math strategies. 

• Statistically significant gains appeared among participants working at the two highest levels of 
reading comprehension but not for those working on level one. Among those working at level 
one, statistically significant gains appeared in decoding scores. Gains in reading ability were 
perceived by participants, teachers, and supervisors/co-workers. 

• Statistically significant gains appeared among participants working on improving their writing. 
Gains in writing ability were also perceived by participants, teachers, and supervisors/co- 

- - workers. - 

• Statistically significant gains appeared among participants working on improving their math. 

Gains in math ability were also perceived by participants and teachers, but not by 
supervisors/co-workers. 7 

• Statistically significant gains appeared among participants working on improving their work- 
related communication skills and problem solving. Gains in work-related communication skills 
and problem solving were perceived by participants and teachers, but not by supervisors/co- 
workers. 

• Participants perceived they had made “good” progress on their top three learning goals during 

both the first six-month and the second six-month period. “ 

• Participants and supervisors perceived increases in a wide range of areas associated with 
positive organizational outcomes. The greatest increase appears to have occurred in 
participants’ interest in continuing their education. 

In summary, the BEST project has met the major goals established at the beginning for improving 
literacy levels (reading, writing, math, communication and problem solving) and demonstrating 
that participation in the BEST program results in positive organizational outcomes. Many lives 
have been fundamentally improved by the results of this program; many positive changes have 
taken place within the organizations in which participants work. 
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^ ^ y- ; Major Assumptions Used to Guide the Evaluation Plan: 



1 . We have grounded teaching/leaming/evaluation in the unique context of each workplace 
and the unique needs of participants. 

~2: We have aggregated data across workplaces. 

3. Key participants were involved in the development of the evaluation. 

4. Objective data on learning outcomes are triangulated with phenomenological data from 
both teachers and participants. 

5. Objective data on workplace outcomes are triangulated with phenomenological data 

from both participants and supervisors/co-workers. " 

6. The Year ^ evaluation includes summative data to determine the extent to which the 
project met its stated outcome goals and objectives. 
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Summative Data: Evaluating Objectives For The BEST Project 

Objective 1. Demonstrate statistically significant gains in participants* awareness 
and application of effective reading, writing, math, communication 
....... ^ problem solving strategies for. the workplace. 

Summary 

- Statistically significant gains appeared on the strategy inventories for participants who worked on 
reading strategies, communication/problem solving strategies, and writing strategies, but not for 
participants who participated on math strategies. Both teachers and participants reported seeing 
gains in reading, writing, math, communication and problem solving strategies for participants 
-who worked in these areas. Supervisors/co-workers perceived gains in participants’ ability to use 
reading, writing, and in their ability to communicate effectively. They did not report gains in 
participants’ ability to use math to solve job-related problems. 

Method 

We evaluated the statistical significance of pre-post gains on each of the four strategy 
inventories: reading, writing, math, communication/problem solving. We then triangulated these 
data with phenomenological data from the Participant Self-evaluation Form, the Teacher’s 
Perception of Improvement Form, and the Changes in Job Performance: Supervisor’s/Co- 
worker’s Response Form. 

Results 

Reading Strategies 

Of the 31 participants who worked on reading strategies and completed both pre and post 
reading' strategy inventories, mean reading strategy scores improved from 6.95 to 1 1.85. This is a 
statistically significant gain (p <.05). 

On average, participants perceived their ability to use reading strategies to have increased as 
well as their ability to figure out new reading strategies that work for them. Mean scores on these 
two items of the Participant Self-evaluation Form were 2.93 and 2.95, respectively (0=greatly 
decreased, l=decreased, 2=no change, 3=increased, 4=greatly increased). 

On average, teachers perceived these participants’ ability to use reading strategies to have 
increased and their ability to figure out new reading strategies that work for them to have also 
increased. Mean scores on these two items of the Teacher’s Perception of Improvement Form 
were 3.21 and 2.85, respectively (0=greatly decreased, l=decreased, 2=no change, 3=increased, 
4=greatly increased). 

On average, supervisors/co-workers perceived these participants’ ability to read job-related 
material to have increased. The mean score on this item of the Changes in Job Performance: 
Supervisor’s/Co-worker’s Response Form was 2.71 (0=greatly decreased, l=decreased, 2=no 
change, 3=increased', 4=greatly increased). The highest score among these participants on the 
Changes in Job Performance: Supervisor’s/Co-worker’s Response Form was on Item 5, “This 
person’s self-confidence” where the mean score was 2.93 (0=greatly decreased, l=decreased, 
2=no change, 3=increased, 4=greatly increased). 

Writing Strategies 

Of the 25 participants who worked on writing strategies and completed both pre and post 
writing strategy inventories, mean writing strategy scores improved from 7.77 to 10.45. This is a 
statistically significant gain (p >.05). 

On average, participants perceived their ability to use writing strategies to have increased and 
their ability to figure out new writing strategies that work for them to have also increased. Mean 
scores on these two items of the Participant Self-evaluation Form were 3. 1 1 and 3. 12, respectively 
(0=greatly decreased, l=decreased, 2=no change, 3=increased, 4=greatly increased). 

On average, teachers perceived these participants’ ability to use writing strategies to have 
increased and their ability to figure out new writing strategies that work for them to have also 
increased. Mean scores on these two items of the Teacher’s Perception of Improvement Form 
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were 3; 12 jnd 3.Q9, respectively (0=greatly decreased, l=decreased, 2=no change, 3=increased, 
4=greatly increased). 

On average, supervisors/co-workers perceived these participants’ ability to write job-related 
material to have increased. The mean score on this item of the Changes in Job Performance: 
Supervisor’ s/Co-worker’ s Response Form was 2.67 (0=greatly decreased, l=decreased, 2=no 
‘ T scores among these participants on the 
Changes in Job Performance: Supervisor’s/Co-worker’s Response Form was on Item 14, “This 
.person’s interest in continuing their education” where the mean score was 3.31 and on item 5 “This 
persons self-confidence” where the mean score was 3.22 (0=greatly decreased, l=decreased, 2=no 
change, 3=increased, 4=greatly increased). 

- - _ - ... Math Strategies 

‘ ' ' Of the 8 participants who worked on math strategies and completed both pre and post math 

strategy inventories, mean math strategy scores improved from 5.63 to 8.55. This is not a 
statistically significant gain (p >.05). 

On average, participants perceived their ability to use math strategies to have increased and 
their ability to figure out new math strategies that work for them to have also increased. Mean 
scores on these two items of the Participant Self-evaluation Form were 3.53 and 3.73, respectively 
(0=greatly decreased, l=decreased, 2=no change, 3=increased, 4=greatly increased). 

On average, teachers perceived these participants’ ability to use math strategies to have 
increased and their ability to figure out new math strategies that work for them to have also 
increased. Mean scores on these two items of the Teacher’s Perception of Improvement Form 
were 3.63 and 3.18, respectively (0=greatly decreased, l=decreased, 2=no change, 3=increased, 
. 4=greatly increased). . 

On average, supervisors/co-workers perceived these participants’ ability to use math to solve 
job-related problems not to have changed. The mean score on this item of the Changes in Job 
Performance: Supervisor’ s/Co-worker’ s Response Form was 2.10 (0=greatly decreased, 
l=decreased, 2=no change, 3=increased, 4=greatly increased). The highest score among these 
participants on the Changes in Job Performance: Supervisor’ s/Co-worker’ s Response Form was 
on Item 5, “This persons self-confidence” where the mean score was 3.1 1 (0=greatly decreased, 
... l=decreased, 2=no change, 3=increased, 4=greatly increased). 

Communication and Problem Solving Strategies 

Of the 6 participants who worked on communication and problem solving strategies and 
completed both pre and post communication and problem solving strategy inventories, mean 
communication and problem solving strategy scores improved from 16.12 to 23.22. This is a 
statistically significant gain (p <.05). 

On average, participants perceived their ability to use communication and problem solving 
strategies to have increased and their ability to figure out new communication and problem solving 
strategies that work for them to have also increased. Mean scores on these two items of the 
Participant Self-evaluation Form were 3.11 and 3.05, respectively (0=greatly decreased, 
l=decreased, 2=no change, 3=increased, 4=greatly increased). 

On average, teachers perceived these participants’ ability to use communication and problem 
solving strategies to have increased and their ability to figure out new communication and problem 
solving strategies that work for them to have also increased. Mean scores on these two items of 
the Teacher’s Perception of Improvement Form were 3.63 and 3.31, respectively (0=greatly 
decreased, l=decreased, 2=no change, 3=increased, 4=greatly increased). 

On average, supervisors/co-workers perceived these participants’ ability to communicate 
effectively at work to have slightly increased. The mean score on this item of the Changes in Job 
Performance: Supervisor’s/Co-worker’s Response Form was 2.42 (0=greatly decreased, 
l=decreased, 2=no change, 3=increased, 4=greatly increased). The highest score among these 
participants on the Changes in Job Performance: Supervisor’s/Co-worker’s Response Form was 
on Item 14, “This persons interest in continuing their education” where the mean score was 3.1 1 
(0=greatly decreased, l=decreased, 2=no change, 3=increased, 4=greatly increased). 
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Objective 2. Demonstrate statistically significant gains in participants’ reading 
comprehension of workplace materials. 



Summary 

ir .: *** V rrrr-S^T-*^'*'- 'glQns''s^pe:aut^d''amori^ pafticiipants working at the two highest levels of 
reading comprehension but not for those working on levels one. Among those working at level 
.... . one, statistically significant gains did appear in decoding scores. 

. On average, participants and teachers perceived participants’ ability in three reading areas to 
have increased: their ability to understand what they read, the extent to which they take control of 
their own learning and reading, and their ability to use reading strategies. Supervisors/co-workers 
- : .-_w perceived gains in participants’ ability to read job related material. 

Method 

We evaluated the statistical significance of pre-post gains on both the decoding and 
comprehension portions of the reading inventory. We then triangulated these data with 
phenomenological data from the Participant Self-evaluation Form, the Teacher’s Perception of 
Improvement Form, and the Changes in Job Performance: Supervisor’s/Co-worker’s Response 
Form. 

Results 

Of the 14 participants who worked on decoding and completed both pre and post reading 
inventories, mean scores on the decoding portion improved from 29.50 to 37.5. This is a 
statistically significant gain (p <.05). 

- Of the 5 participants who worked on reading and completed Level 1 Comprehension pre and 

post reading inventories, mean scores on Level 1 Comprehension improved from 6.48 to 7.47. 
This is not a statistically significant gain (p <.05). 

Of the 10 participants who worked on reading and completed Level 2 Comprehension pre and 
post reading inventories, mean scores on Level 2 Comprehension improved from 5.02 to 7.00. 
This is a statistically significant gain (p <.05). 

Of the 1 6 participants who worked on reading and completed Level 3 Comprehension pre and 
.post reading inventories, mean scores on Level 3 Comprehension improved from 3.21 to 4.64. 
This is a statistically significant gain (p <.05). 

. . On average, participants perceived their ability in three reading areas to have increased: their 
ability to understand what they read, the extent to which they take control of their own learning and 
reading, and their ability to use reading strategies. Mean scores on these three items of the 
Participant Self-evaluation Form were 3.30, 3.14, and 3.25, respectively (0=greatly decreased, 
l=decreased, 2=no change, 3=increased, 4=greatly increased). 

. °n average, teachers also perceived these participants’ ability in three reading areas to have 

increased: their ability to understand what they read, the extent to which they take control of their 
own learning and reading, and their ability to use reading strategies. Mean scores on these three 
items of the Teacher’s Perception of Improvement Form were 3.22, 3.05, and 3.06, respectively 
(0=greatly decreased, l=decreased, 2=no change, 3=increased, 4=greatly increased). 

On average, supervisors/co-workers also perceived these participants’ ability to read job- 
related material to have increased. The mean score on this item of the Changes in Job 
Performance: Supervisor’s/Co-worker’s Response Form was 2.78 (0=greatly decreased, 
l=decreased, 2=no change, 3=increased, 4=greatly increased). The highest scores among these 
participants on the Changes in Job Performance: Supervisor’s/Co-worker’s Response Form was 
on Item 14, “This persons interest in continuing their education” where the mean score was 3.44 
and on Item 5, “This person’s self confidence” where the mean score was 3.25 (0=greatly 
decreased, l=decreased, 2=no change, 3=increased, 4=greatly increased). 
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Objective ^..Demonstrate statistically significant gains in participants’ writing for 
the workplace. 



Summary 

Statistically significant gains appeared among participants working on improving their 

participants’ ability to have increased 
in three areas related to writing: their ability to communicate in writing, the extent to which they 
take control of their own learning about writing, and their confidence in their ability to become a 
better writer. Supervisors/co-workers also perceived gains in participants’ ability to write job 
related material. In addition, supervisors/co-workers also perceived gains in participants’ interest 
in continuing their education and in self-confidence. 



Method 

We evaluated the statistical significance of pre-post gains on the writing sample. We then 
triangulated these data with phenomenological data from the Participant Self-evaluation Form, the 
Teacher’s Perception of Improvement Form, and the Changes in Job Performance: 
Supervisor’s/Co-worker’s Response Form. 



Results 

Of the 33 participants who worked on writing and completed both pre and post writing 
inventories, mean scores on writing sample improved from 6.84 to 8.49. This is a statistically 
significant gain (p <.05). 

On average, participants perceived their ability in three writing areas to have increased: their 
ability to communicate in writing, the extent to which.they take control of their own learning about 
writing, and their confidence in their ability to become a better writer. Mean scores on these three 
items of the Participant Self-evaluation Form were 2.98, 3.21, and 3.22, respectively (0=greatly 
decreased, l=decreased, 2=no change, 3=increased, 4=greatly increased). 

On average, teachers also perceived these participants’ ability in three writing areas to have 
increased: their ability to communicate in writing, the extent to which they take control of their own 
.learning about writing, and their confidence in their ability to become a better writer. Mean scores 
on these three items of the Teacher’s Perception of Improvement Form were 2.88, 3.1 1, and 3. 15, 
respectively (0=greatly decreased, l=decreased, 2=no change, 3=increased, 4=greatly increased). 

On average, supervisors/co-workers also perceived these participants’ ability to write job- 
related material to have increased. The mean score on this item of the Changes in Job 
Performance: Supervisor’s/Co-worker’s Response Form was 2.77 (0=greatly decreased, 
l=decreased, 2=no change, 3=increased, 4=greatly increased). The highest scores among these 
participants on the Changes in Job Performance: Supervisor’s/Co-worker’s Response Form was 
on Item 14, “This person’s interest in continuing their education” where the mean score was 3.23 
and on Item 5, ‘This person’s self confidence” where the mean score was 3.17.” (0=greatly 
decreased, l=decreased, 2=no change, 3=increased, 4=greatly increased). 



Page 9 



- v Objective^ 4rPemonstrate statistically significant gains in participants’ work- 
related computational skills and mathematical problem solving. 

Summary 

Statistically significant gains appeared among participants working on improving their math. 
' ; V feachersalso'pereeived participants ’ abilityto have increased in three 

math areas to have increased: their ability to use math to solve problems, the extent to which they 
take control of their own learning about math, and their confidence in their ability to become better 
at math. On average, supervisors/co-workers did not perceive these participants’ ability to use math 

to solve job-related problems to have increased substantially. However, supervisors/co-workers 
perceived gains in participants’ self-confidence and their ability to be responsible for their own 
...... .learning at_work. 



Method 

We evaluated the statistical significance of pre-post gains on the Math Inventory. We then 
triangulated these data with phenomenological data from the Participant Self-evaluation Form, the 
Teacher’s Perception of Improvement Form, and the Changes in Job Performance: 
Supervisor’ s/Co-worker’s Response Form. 



Results 

Of the 15 participants who worked on math and completed both pre and post math 
inventories, mean scores on math improved from 5.39 to 8.91. This is a statistically significant 
gain (p <.05). 

On average, participants perceived their ability in three math areas to have increased: their 
ability to use math to solve problems, the extent to which they take control of their own learning 
about math, and their confidence in their ability to become better at math. Mean scores on these 
three items of the Participant Self-evaluation Form were 3.05, 2.83, and 3.13, respectively 
(0=greatly decreased, l=decreased, 2=no change, 3=increased, 4=greatly increased). 

On average, teachers also perceived these participants’ ability in three math areas to have 
increased: their ability to communicate in writing, the extent to which they take control of their own 
learning about writing, and their confidence in their ability to become a better writer. Mean scores 
on these three items of the Teacher’s Perception of Improvement Form were 3.15, 2.89, 2.84, 
respectively (0=greatly decreased, ^decreased, 2=no change, 3=increased, 4=greatly increased). ’ 

On average, supervisors/co-workers did not perceive these participants’ ability to use math to 
solve job-related problems to have increased substantially. The mean score on this item of the 
Changes in Job Performance: Supervisor’s/Co-worker’s Response Form was 2.22 (0=greatly 
decreased, l=decreased, 2=no change, 3=increased, 4=greatly increased). The highest scores 
among these participants on the Changes in Job Performance: Supervisor’ s/Co-worker’s Response 
Form was on Item 5, “This person’s self confidence” where the mean score was 3.04 and on Item 
6 “This person’s ability to be responsible for their own learning at work.” where the mean score 
was also 3.11 (0=greatly decreased, l=decreased, 2=no change, 3=increased, 4=greatly 
increased). 
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. Objective. 5.. Demonstrate statistically significant gains in participants’ work- 
related communication skills and problem solving. 

Summary 

Statistically significant gains appeared among participants working on improving their work- 
related ctommuffi^^ and problemsolving. Off average, participants and teachers perceived 

participants’ ability to have increased in three areas related to communication and problem solving 
skills: their ability to communicate and solve problems effectively, the extent to which they take 
control of their own learning about communication and problem solving, and their ability to figure 
out new communication and problem solving strategies that work for them. Supervisors/co- 
workers did not perceive these participants’ ability to communicate effectively at work to have 
increased substantially. However, supervisors/co-workers perceived gains in participants’ interest 
in continuing their own education. 



Method 

We evaluated the statistical significance of pre-post gains on the Interpersonal Communication 
and Problem Solving Inventory. We then triangulated these data with phenomenological data from 
the Participant Self-evaluation Form, the Teacher’s Perception of Improvement Form, and the 
Changes in Job Performance: Supervisor’s/Co-worker’s Response Form. 

Results 

Of the 14 participants who worked on communications and problem solving and completed 
both pre and post communications and problem solving inventories, mean scores improved from 
77.59 to 90.23. This is a statistically significant gain (p <.05). 

On average, participants perceived their ability in three communication and problem solving 
areas to have increased: their ability to communicate and solve problems effectively, the extent to 
which they take control of their own learning about communication and problem solving, and their 
ability to figure out new communication and problem solving strategies that work for them. Mean 
scores on these three items of the Participant Self-evaluation Form were 3.05, 3.15, and 3.21, 
respectively (0=greatly decreased, l=decreased, 2=no change, 3=increased, 4=greatly increased). 

On average, teachers also perceived these participants’ ability in three communication and 
problem solving areas to have increased: their ability to communicate and solve problems 
effectively, the extent to which they take control of their own learning about communication and 
problem solving, and their ability to figure out new communication and problem solving strategies 
that work for them. Mean scores on these three items of the Teacher’s Self-evaluation Form were 
3.39, 3.45, and 3.65, respectively (0=greatly decreased, l=decreased, 2=no change, 3=increased, 
4=greatly increased). 

On average, supervisors/co-workers did not perceive these participants’ ability to 
communicate effectively at work to have increased substantially. The mean score on this item of the 
Changes in Job Performance: Supervisor’s/Co-worker’s Response Form was 2.40 (0=greatly 
decreased, l=decreased, 2=no change, 3=increased, 4=greatly increased). On average, 
supervisors/co-workers also did not perceive these participants’ ability to solve problems at work 
to have increased substantially. The mean score on this item of the Changes in Job Performance: 
Supervisor’s/Co-worker’s Response Form was 2.20 (0=greatly decreased, l=decreased, 2=no 
change, 3=increased, 4=greatly increased). The highest score among these participants on the 
Changes in Job Performance: Supervisor’s/Co-worker’s Response Form was on Item 14, ‘This 
person’s interest in continuing their education” where the mean score was 3.00 (0=greatly 
decreased, l=decreased, 2=no change, 3=increased, 4=greatly increased). 
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^ Ol^ectiTe 6.pocument achievement of individual learning goals. 

Summary 

Participants perceived they had made “good” progress on their top three learning goals 
throughout the program. 

Method 

We evaluated the achievement of individual learning goals with the Learning Goals Inventory, 
this allowed participants to define their individual learning goals each six months and then evaluate 
the extent to which they made progress on each goal. 

. ... . .. . . . . Results 

- ’ On average, participants perceived they had made “good” progress on their top three learning 

goalsthroughout the program. Mean scores on these three items were 2.99, 2.78, and 2.76 
(0=none, l=a little, 2=some, 3=good, 4=great). 

On average, participants perceived they had made “good” progress on their top three learning 
goals throughout the program. Mean scores on these three items were 2.91, 2.88, and 2.77 
(0=none, l=a little, 2=some, 3=good, 4=great). 
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Objectively? Demonstrate that completion of the BEST program results in positive 
organizational outcomes such as improved employee performance, 
especially increased involvement in activities designed for 
continuous quality improvement. 



-- . . 

Participants and supervisors perceived increases in a wide range of areas associated with 
positive organizational outcomes. The greatest increase appears to have occurred in participants 
' interest in continuing their education 



Method 

_ We evaluated attaining positive organizational outcomes by looking at the perceptions of both 
participants and their supervisors/co-workers in several areas: 

• their ability to work effectively with a team; 

• their initiative at work; 

• their knowledge about the job they do; 

• the quality of their work; 

• the quantity of their work; 

• the responsibility they assume in their work; 

• their interest in continuing their education; 

• their willingness to accept changes at work; 

• their being respectful of diversity and other points of view at work; and 

• their involvement in activities designed for continuous quality improvement. 



Each item was rated on a 0-4 scale (0=greatly decreased, l=decreased, 2=no change, 3=increased, 
4=greatly increased). 

_ . Results 

The average value for each item by participants and supervisors/co-workers is reported below. 
Participants and supervisors perceived increases in a wide range of areas associated with positive 
organizational outcomes. The greatest increase appears to have occurred in participants’ interest in 
continuing their education. This item was rated highest by both groups. 

Item Participants Supervisors 



Ability to work effectively with a team. 


2.90 


2.68 


Initiative at work 


2.94 


2.55 


Knowledge about the job they do 


3.08 


2.45 


Quality of their work 


2.91 


2.44 


Quantity of their work 


2.86 


2.47 


Responsibility they assume in their work 


2.95 


2.67 


Interest in continuing their education 


3.45 


2.99 


Willingness to accept changes at work 


2.98 


2.78 


Being respectful of diversity and other points of 
view at work 


2.92 


2.47 


Involvement in activities designed for 
continuous quality improvement 


2.77 


2.85 
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Major Assumptions Used to Guide the Development of the 

Evaluation Plan: 

1 . We want to ground teaching/leaming/evaluation in the unique context of each 

" V ; ' v ‘ ^ wtjivkpla^ of-participafvts. 

2 . We want to be able to aggregate data across workplaces. 

3 . Key participants should be involved in the development of the evaluation. 

4. Objective data on learning outcomes should be triangulated with phenomenological data 

- from both teachers and participants. 

5. Objective data on workplace outcomes should be triangulated with phenomenological 
data from both participants and supervisors/co-workers. 

6. The evaluation should include: 

• formative data to inform teachers, participants, and business partners about the 
quality of each learning experience provided to participants; 

• summative data to determine the extent to which the project met its stated 
outcome goals and objectives; and 
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" Formative Data: Evaluating the Quality of Each Learning 

Experience 

Course Evaluation Form (See page 8) 

provide instructors, -participants, - and - business partners with 
information about the quality of each learning experience provided to participants. 
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^ ^ ; Summative Data: Evaluating Outcome Goals and Objectives 

These goals and objectives come from the original grant proposal. The objectives are stated as 
questions: 

•' - ‘’ r - GoaTi^ ^."Iffcie^TKe'Btei^y s lev^^ individuals especially in regards to reading, 

writing, math, communication and problem solving tasks in the workplace. 

1.1 How will we demonstrate statistically significant gains in participant’s 
awareness and application of effective reading, writing, math, communication 
and problem solving strategies for the workplace? 

■ • Triangulated data points: 

• Pre-post comparisons of mean scores on strategy 
inventories using ANOVA procedures to determine 
statistical significance. (See pages 9- 1 2) 

• Participant Self-evaluation Form (Items A 4-5, B 4-5, C 
4-5, D 4-5 on pages 13-15) 

• Teacher’s Perception of Improvement Form (Items A 4- 
5, B 4-5, C 4-5, D 4-5 on pages 16-17) 

1.2 How will we demonstrate statistically significant gains in participant’s 
reading comprehension of workplace materials? 

. ...... ... .. “ Triangulated data points: 

• Pre-post comparisons of mean scores on reading 
comprehension measures using ANOVA procedures to 
determine statistical significance. (See pages 1 8-25) 

• Participant Self-evaluation Form (Items A 1 -3 on pages 
- 13-15) 

• Teacher’s Perception of Improvement Form (Items A 1- 
3 on pages 16-17) 

• Changes in Job Performance: Supervisor’ s/Co- 

worker’s Response Form (Items 1, 5, 6 on pages 26- 
27) 

1.3 How will we demonstrate statistically significant gains in participant’s 

composition (writing) for the workplace? 

Triangulated data points: 

• Pre-post comparisons of mean scores on writing sample 
using ANOVA procedures to determine statistical 
significance (See page 29) 

• Participant Self-evaluation Form (Items B 1-3 on pages 
13-15) 

• Teacher’s Perception of Improvement Form (Items B 1- 
3 on pages 16-17) 

• Changes in Job Performance: Supervisor’s/Co- 

worker’s Response Form (Items 2, 5, 6 on pages 26- 
27) 
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... .......... .. _ .. 1.4. How will we demonstrate statistically significant gains in participant’s work- 

related computational skills and mathematical problem solving? 

Triangulated data points: 

• Pre-post comparisons of mean scores on Math 

. ^ .foventQry_ using ANQVA ..procedures to determine 

statistical significance (Seepages 29-31) 

• Participant Self-evaluation Form (Items C 1-3 on pages 
13-15) 

• Teacher’s Perception of Improvement Form (Items C 1- 
3 on pages 16-17) 

• Changes in Job Performance: Supervisor’s/Co- 

worker’s Response Form (Items 3, 5, 6 on pages 26- 
27) 

1.5 How will we demonstrate statistically significant gains in participant’s work- 
related communication skills and problem solving? 

Triangulated data points: 

• Pre-post comparisons of mean scores on Interpersonal 
Communications and Problem Solving Inventory using 
ANOVA procedures (See pages 34-38) 

• Participant Self-evaluation Form (Items D 1-3 on pages 
13-15) 

• Teacher’s Perception of Improvement Form (Items D 1- 
3 on pages i 6- 17) 

• Changes in Job Performance: Supervisor’s/Co- 

worker’s Response Form (Items 4, 13, 5, 6 on pages 
26-27) 

1.6 How will we document participant achievement of individual learning goals? 

• Learning Goals Inventory (See page 39) 

Goal 2. Demonstrate that completion of the BEST program results in positive organizational 
outcomes such as improved employee performance, especially increased involvement in 
activities designed for continuous quality improvement 

2. 1 How will we demonstrate that supervisors report gains in the job performance 
of employees who complete the BEST program, that employee self-reports 
will demonstrate gains in job performance, and that positive changes take place 
with other measures regarded by firm leaders as salient criteria? 

Triangulated data points: 

• Changes in Job Performance: Supervisor’s/Co- 

worker’s Response Form (Items 7-12, 14-17 on pages 
26-27) 

• Participant Self-evaluation Form (Items E 7-12, 14-17 
on pages 13-15) 



Course Evaluation Form - B.E.S.T. 

Date: 

Company: 

Course: Title: - . ••• — ••••-7 — • ___ : - - ■ -- 

Check one: 

□ Workshop 

□ Brush-up 

□ Communication and problem solving course 

Purpose: — The BEST program would like to know what you think about the quality of your 
learning experience. Your honest and objective responses will help us to improve 
our instruction. Thank you. 

Directions: Participants: Please mark the box that best represents your thinking about each item. 
Use the following scale. ( Teachers : Please attach a cover page for each class, 
workshop, or individual set of Course Evaluation Forms indicating your estimate of 
the percentage of time spent on teaching, evaluation, and combined teaching - 
evaluation. Include this with the packet you submit to the business partner and the 
project director. ) 



The Instructor 


Low 


Moderate 
1 2 3 


High 

4 


1 . Demonstrated knowledge of the subject. 


□ 


□ 


□ 


□ 


□ 


2. Presented the material clearly. 


□ 


□ 


□ 


□ 


□ 


3. Treated me with respect. 


□ 


□ 


□ 


□ 


□ 


4. Created a learning environment that was non-threatening. 
Participants 


□ 


□ 


□ 


□ 


□ 


1 . Were encouraged to ask questions and participate in discussions. □ 


□ 


□ 


□ 


□ 


2. Were encouraged to set goals for themselves. 


□ 


□ 


□ 


□ 


□ 


3. Were supportive of each other. 


□ 


□ 


□ 


□ 


□ 


Myself 


1 . I developed more confidence about my ability to learn. 


□ 


□ 


□ 


□ 


□ 


2. I increased my ability to learn by myself. 


□ 


□ 


□ 


□ 


□ 


3. I would recommend this course to my co-workers. 


□ 


□ 


□ 


□ 


□ 



Other 

1. How have you used what you learned in this course on the job? at home? in the community? 



2. How can we improve this course for the next group of participants? Please share your ideas. 



■ Reading Strategies Inventory - B.E.S.T. 

Name: Date: 

Company: Course: 

Purpose: This is a chance to find out what you do as you read. The information in this form 

will only be used in the BEST program. It will not be used by your company to 
evaluate employees. 

Directions: Participants may choose to complete this in writing or orally. As you consider each 
_ ..... . part of the reading activity below, tell us all of the things you would do to help you 
complete the task successfully. 



Part 1: The Situation and Beginning Strategies 

You are beginning a new job at your company. Your supervisor just gave you a handbook 
describing all of your work responsibilities. She says to read this handbook and then she wants to 
talk to you about your job. What strategies would you use before reading the entire handhnnk 
carefully? Make a list and number each item. 



Part 2: In the Middle 

The handbook gets a little hard to read at some locations. While vou are reading these parts, what 
strategies would you use to help you understand the information? Make a list and number each 
item. 



Part 3: At the End 

You have finished reading all of the manual. Before vou talk to vour supervisor, what strategies 
would vou use to be certain you understand everything? Make a list and number each item. 
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. . .-- Writing Strategies Inventory - B.E.S.T. 

Name: Date: 

Company: _ ..... Course 

.... -rv . Purpose: This is a chance to find out what you do as you write. The information in this form 

will only be used in the BEST program. It will not be used by your company to 
evaluate employees. 

- .... .... .Directions: Participants may choose to complete this in writing or orally. After completing the 

* writing sample you should also complete this form. Tell us all of the things you did 

to help you successfully complete the writing task. 



Part 1: Beginning Strategies 

Before you began the writing task, what did you do? Write down all of the strategies you used 
before you began to write. Begin each item on a new line. 



Part 2: In the Middle 

When you were in the middle of writing the description of your job, what did you do? Write down 
all of the strategies vou used as you wrote this piece. Begin each item on a new line. 




Part 3: At the End 

When you finished writing a description of your job, what did you do? Write down all of the 
strategies you used at the end of your writing. Begin each item on a new line. 
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Math Strategies Inventory 



B.E.S.T. 



Name: Date: 

v ^ C6imOah ^'' ; " : ^^ ' ~ ' - ‘ _ ' r ~.~ Course: 

- . Purpose: This is a chance to find out what you do as you solve math problems. The 

information in this form will only be used in the BEST program. It will not be used 
by your company to evaluate employees. 

~ >;••• - -Directions: Participants may choose to complete this in writing or orally. As you consider each 

part of the math activity below, tell us all of the things you would do to help you 
successfully complete the math task. 

Part 1: The Situation and Beginning Strategies 

Identify a situation where you have to use math to solve a really difficult problem at work. 
Describe the situation below. Now, think about what you would do before beginning to solve this 
problem. Write down the strategies you would use before vou actually started to solve this 
problem. Make a list and number each item. * 



Part 2: In The Middle 

...You have started to work on the problem you described above, but you find yourself stuck. What 
you are doing just doesn’t seem right. What strategies would vou use now? Write down 
everything vou might do to solve the problem . Begin each item on a new line. 



Part 3: At the End 

You think you have come up with a solution to this challenging math problem. Before you use the 
answer in your work, is there anything else that you would do? Write down all of the strategies 
vou would use after solving this problem but before using the answer in vour work . Begin each 
item on a new line. 



O 

ERLC 
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Communication and Problem Solving Strategies Inventory 

B.E.S.T. 



Name: Date: 

Company: Course: 

Purpose: This is a chance’ to find out what you do as you communicate and solve problems at 

work. The information in this form will only be used in the BEST program. It will 
not be used by your company to evaluate employees. 

Directions: Participants may choose to complete this in writing or orally. As you consider each 
part of the activity below, tell us all of the things you would do to help you 
successfully complete the communication and problem solving task. 

Part 1: The Situation and Beginning Strategies 

You have a problem at work that involves several individuals with whom you work. Describe the 
situation below. Now, think about the strategies you would use at the beginning as you first start 
to think about this problem. Write down everything you would do as vou first begin to think about 
solving the problem. Make a list and number each item. 



Part 2: In the Middle 

You are in the middle of working with your colleagues to solve this problem that involves all of 
-you. As you work through the problem, what types of strategies would you use to be certain you 
are communicating effectively and solving the problem correctly? Write down everything vou 
would do. Make a list and number each item. 



Part 3: At the End 

You have finished solving this problem. Before you leave your colleagues, are there any strategies 
you would use? Write down everything vou would do. Make a list and number each item. 
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Participant Self-evaluation Form - B.E.S.T. 



Name: 



Date: 



Comjpanv r^ T " _ - ' •••• Course: 



Purpose: This form will help us to determine if you see changes in your learning. The 

information in this form will only be used in the BEST program. It will not be used 
by your company to evaluate employees. 

Directions: The participant should complete the information below. Afterwards, the 
information should be shared in a conference with the teacher. 



Changes since beginning Amount of Change 

participation in this program 





greatly 

decreased 


decreased 


no 

change 


increased 


greatly 

Increased 


dont 

know 


continues 
to be 
excellent 


does f 
apply 


A. 

1 . 


Reading 

My ability to understand what 1 read. □ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


2. 


The extent to which 1 take control of 


□ 


a 


□ 


□ 


□ 


□ 




3. 


own learning about reading. 

My confidence in my ability to becomEh 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


4. 


better reader. 

My ability to use reading strategies. □ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


5. 


My ability to figure out new reading Q~ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


B. 

1 . 


strategies that work for me. 

Writing 

My ability to communicate in writing. □ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


2. 


The extent to which 1 take control of 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


3. 


own learning about writing. 

My confidence in my ability to becomEh 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


4. 


better writer. 

My ability to use writing strategies. □ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


5. 


My ability to figure out new writing □ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 



strategies that work for me. 
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greatly 

decreased 


decreased 


no 

change 


Increased 


greatly 

increased 


don't 

know 


continues 
to be 
excellent 


does i 
apply 


C. Math 

1 . My ability to use math to solve probleSs. 


□ 


□ 


□ 


□ ' 


□ 


□ 


□ 


2. " The V>dehT to Which 1 take control of ' 




□ 


□ 


□ 


□ 


□ 


□ 


own learning about math. 

3. My confidence in my ability to becomO 


Q 


□ 


□ 


□ 


□ 


□ 


□ 


better at math. 

4. My ability to use math and problem sfiQing 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


strategies. 

5. My ability to figure out new math and □ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


problem solving strategies that work for 
me. 

D. Communication and Problem 
Solving 

1 . My ability to communicate and solve □ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


problems effectively. 

2. The extent to which 1 take control of 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


own learning about communication and 
problem solving. 

3. . My confidence in myability to commiQcate 


□ 


□ 


□ 


□ 


□ 


□ 


□ ' 


effectively. 

4. My ability to use communication and □ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


problem solving strategies. 

5. My ability to figure out new communiQtion 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


and problem solving strategies that work. 
.... forme. 

E. Other 

1 . . My overall self-confidence in myself alik 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


learner. 

2. The responsibility 1 take for my own □ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


learning. 

3 . My ability to work effectively with a te£di. 


□ 


□ 


□ . 


□ 


□ 


□ 


□ 


Other (please describe) 

4. □ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


5. □ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 
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greatly 

decreased 


decreased 


no 

change 


increased 


greatly 

increased 


don't 

know 


F. Workplace Changes: 












. ,-1. ^My.abHjtyJo^ 






□ 


□ 


□ 


2. My ability to write job-refated material O 


□ 


□ 


□ 


□ 


□ 


.3. . My ability to use math to solve job-relSed 


□ 


□ 


□ 


□ 


□ 


problems. 

4. My ability to communicate effectively 61 


□ 


□ 


□ 


□ 


□ 


work. 












? ‘ My self-confidence. □ 


□ 


□ 


□ 


□ 


□ 


6 . My ability to be responsible for my ovE3 


□ 


□ 


□ 


□ 


□ 


learning at work. 

7. My ability to work effectively with a te£3i. 


□ 


□ 


□ 


□ 


□ 


8. My initiative at work. □ 


□ 


□ 


□ 


□ 


□ 


9. My knowledge about the job 1 do. □ 


□ 


□ 


□ 


□ 


□ 


10. The quality of my work. □ 


□ 


□ 


□ 


□ 


□ 


1 1 . The quantity of my work. □ 


□ 


□ 


□ 


□ 


□ 


1 2. The responsibility 1 assume in my woiO 


□ 


□ 


□ 


□ 


□ 


.. ... ... .. . 1 3. My ability to solve problems at work. Q 


.□ 


□ 


□ 


□ 


□ 


1 4. My interest in continuing my educati<53. 


Q 


Q 


□ 


□ 


□ 


1 5. My willingness to accept changes at lEfcrk. 


□ 


□ 


□ 


□ 


□ 


1 6. My being respectful of diversity and fither 


□ 


□ 


□ 


□ 


□ 


points of view at work. 












1 7. Involvement in activities designed foQ- 
— continuous quality improvement. 


□ 


□ 


□ - 


□ 


□ 


Other (please describe) 

18. □ 


□ 


□ 


□ 


□ 


□ 


19. □ 


□ 


□ 


□ 


□ 


□ 



continues 

to be does nc 

excellent apply 



□ □ 

□ □ 

□ □ 

□ □ 

□ □ 

□ □ 

□ □ 

□ □ 

□ □ 

□ □ 

□ □ 

□ □ 

□ □. 

□ □ 

□ □ 

□ □ 

□ □ 

□ □ 

□ □ 



Feel free to add comments below or on the back of this page, if necessary. 
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Teacher's Perception of Improvement Form - B.E.S.T. 



. Name: _ v .. __ . . _ ; _Date: 

Company:^ Course: 



Purpose: This form will help us to determine if teachers see changes in participants’ 

performance in their learning experiences. The information in this form will only 
be used in the BEST program. It will not be used by your company to evaluate 
employees. 



Directions: The teacher should complete the information below for each participant in the 

BEST program. Afterwards, the information should be shared in a conference 
with the participant. 



Changes since beginning Amount of Change 

participation in this program 



_ — greatly - . 




no 




greatly 


dont 


continues 
to be 


does nc 


- — -■ decreased 

A. Reading 


decreased 


change 


increased 


increased 


know 


excellent 


apply 


1 . This person’s ability to understand wQt 
they read. 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


2. The extent to which this person takeO 
control of their own learning about reading. 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


3. This person’s confidence in their abilG) to 
become a better reader. 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


4. This person’s ability to use reading □ 
strategies. 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


5. This person’s ability to figure out reaflihg 
strategies that work for them. 

B. Writing 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


1 . This person’s ability to communicate Q 
writing. 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


2. The extent to which this person takeQ 
control of their own learning about writing. 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


3. This person’s confidence in their abilG^ to 
become a better writer. 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


4. This person’s ability to use writing □ 
strategies. 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


5. This person’s ability to figure out writing 
strategies that work for them. 


□ 


□ 


□ 


□ 


□ 


□ 


□ 
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continues 



greatly 

decreased 


decreased 


no 

change 


increased 


greatly 

increased 


donl 

know 


to be 
excellent 


does nc 
apply 


C. Math 

1 . This person’s ability to use math to s<EB r e 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


















2. The extent to which this person takeO 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


control of their own learning about math. 
3. This person’s confidence in their ability to 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


become better at math. 
















4. This person’s ability to use math andQ 
problem solving strategies. 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


5 . This person’s ability to figure out matSfend 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


problem solving strategies that work for 
them. 
















D . Communication and Problem 
















Solving 

1 . This person’s ability to communicate Slid 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


solve problems effectively. 

2. The extent to which this person takeQ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


control of their own learning about 
communication and problem solving. 
















.. - ,3. This' person’s confidence in their abilEJ to 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


communicate effectively. 
















4. This person’s ability to use communiOtion 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


and problem solving strategies. 

5. This person’s ability to figure out □ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


communication and problem solving . . 
















— strategies that work for them. 
















E . Other 

1 . This person’s overall self-confidenceOi 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


themself as a learner. 
















2. The responsibility this person takes £3 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


their own learning. 

3. This person’s ability to work effectivelylwith 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


a team. 
















Other (please describe) 

4. □ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


5. □ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 
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Reading Inventory - B.E.S.T. 



Name: Date: 

£ompahv r x ' ------v-- ■- - Course: 



Purpose: These activities will help us both to understand your strong areas in reading. The 

information in this form will only be used in the BEST program. It will not be used 
by your company to evaluate employees. 



WORD RECOGNITION 

Purpose: The purpose of this activity is to help you to find out which words you can already 

recognize by sight. 

Directions: Read instructions below to participant. Record miscues. Mark each correct word 
response with C. Mark substitutions above the target word. Mark no responses by 
circling the target word. When the teacher needs to pronounce a word, indicate this 
with a T. Total correct word responses at the bottom. If participant recognizes all 
of the words correctly, this portion of the reading inventory need not be repeated. 

I’d like you to try reading each of these words. Tell me when you would 
like to skip a word. 



I. Words in Isolation 



— above 


against 


below 


bottom 


collected 


decide 


during 


factories 


here 


hospital 


office 


phone 


plant 


police 


problem 


were 


when 


where 


worker 


wrote 



II. Words in Context 
I do not think we should do it that way. 



I want to apply for a new position at work. 



Number correct (Please record the score) 
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• UNDERSTANDING WHAT YOU READ - I 

Note: Use the performance on the word list to make a preliminary assessment of the appropriate 
level for reading comprehension. If the participant scores 8 or more (out of 10) on the 
comprehension questions for Level 1 or 2, have them attempt to complete the next highest 

• -yf ■■ •-•••••_- -■- ~tevefc^Begifl*W^ ' • ' 

Purpose: The purpose of- this activity is to help you to find out how well you understand 

what you read. 



Directions: 



Please read the story “My First Deer with Bow and Arrow”. Afterwards, I will ask 
you five questions. You will be able to look back if you wish. 



- Have students read the selection orally or silently. 

- Afterwards, ask the comprehension questions below. 

- Score each answer as correct (2 points), partial (1 point), or 
incorrect (0 points). 

- Score for gist, not word accuracy of response. 

- Allow students to refer back to the passage to answer 
questions. 

- Record total score (10 total points) and time. 



I. My First Deer with Bow and Arrow (0, 1, or 2 points for each item) 

1 . How can you tell that this hunter’s brother is probably an older brother? 
(“My brother had been hunting for a few years.” This hunter was 14 
years old. “I hollered to my brother for help.”) 

2 . Why did he get impatient and move around? 

(He was a younger hunter.) 

3 . Why was it so hard for him to pull the bow back? 

(He was shaking, or He was too young and not strong enough.) 

4. How many arrows did he need to kill this deer? 

(Two) 

5 . Did he kill an older or younger deer? How can you tell? 

(Younger. It was a button horn buck.) 



Total Score 
Time 
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UNDERSTANDING WHAT YOU READ - II 



Purpose: The purpose of this activity is to help you to find out how well you understand 

^ : -'~* v - x ^^whaly6u^ 

Directions: Please read the article “Women and Poverty”. Afterwards, I will ask you five 
questions. You will be able to look back if you wish. 

- Have students read the selection orally or silently. 

_ - - After reading, ask the comprehension questions below. 

- Score each answer as correct (2 points), partial ( 1 point), or incorrect (0 
points). 

- Score for gist, not word accuracy of response. 

- Allow students to refer back to the passage to answer questions. 

- Record the time required for completing each passage. 

- Record total score 



Women and Poverty 



All over the United State, finding decent, affordable housing has become more and more 
difficult for women supporting families on low incomes. 

• About 1/4 of all American families are single-parent families. 

• In 1985, 87 our of every 100 single-parent households were headed by women. 

• There are about 5 1/2 million single mothers. 

• There are about 900,000 single fathers. 

Generally, single mothers have a lot less money than single fathers. In 1985 the average 
income for single mothers was $1 1,644; average income for single fathers was $23,368. In 1986 
there were 1 1,944,000 people living in female-headed families who were poor. 

In fact, most women, even those who aren’t single parents, don’t earn as much as men. In 
1988 women working full time earned 70% of what men earned. That means for every dollar a 
man earned, she earned 70 cents. The difference is even greater for workers with college degrees. 
Female college graduates earned 59% of what male college graduates earned. 

In 1986, out of every 100 families headed by an employed woman, 22 were “poor”, according 
to government definitions. 
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I. Women and Poverty (0, 1, or 2 points for each item) 

1 . Why is it so difficult for single women to find affordable housing? 
(They make less money than men.) 



2. What percentage of families in America have only one parent at home 7 

(25%) 

3 . On average, how much more did single fathers make compared to single 
mothers in 1985? 

(about $ 12,000 or twice as much) 



4. In 1988, how much did women working full time earn compared to men? 
(About 70% of what men earned.) 



5 . In 1 988, for every dollar a male college graduate earned how much did a 
female college graduate earn? 

(Fifty-nine cents) 






Total Score 
Time 



r 



O 

ERIC 
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UNDERSTANDING WHAT YOU READ III-A 

Purpose: The purpose of this activity is to help you to find out how well you understand 

what you read. 

r? ' : : ; Directions:^ Please read both articl'es.' "After each article, I will ask you five questions. You will 

be able to look back if you wish. 

. . . ■ - * - Have students read the two s elections orally or silently 

- After reading each selection, ask the comprehension questions below. 

- Score each answer as correct (1 points), partial (1/2 points), or incorrect (0 
points). 

• “ ~ ■ ' ' r' ------ — ■ -- ■ - Score for gist, not word accuracy of response. 

- Allow students to refer back to the passage to answer questions. 

- Record the time required for completing each passage. 

- Record total score 



Beliefs, Principles, and Values 

Tolerance 

Tolerance is the practice of allowing or respecting the nature, beliefs or behaviors of others. 
Both inside and outside the company, tolerance can only enhance the achievement of our mission. 

Personal Growth 

This company is about personal growth. Whether it is growth in job responsibility, pay, 
education, or psychological and spiritual growth, we need to encourage it all! By the same token, 
there are a lot of people who are very content doing exactly what they have been doing and strictly 
working for their wages. This has to be OK, too. Along with growth within the company, we 
need to make every effort to promote personal growth when we bring new people into the 
organization. It would become a major part of our hiring policy, a kind of affirmative action 
program for anyone in need. 

Individuality 

Let me stress once again that the whole system being developed in this plan is based on 
improving the quality of life for the individual. Individuality is the strength of this organization 
and it is revered. The only time individuality takes second place is when the individual demand is 
counter to the need of the group; i.e. the common good. 

Motivation 

What makes me do something? People work for a variety of reasons, but motivators can 
essentially be labeled a sjnaterial and emotional. Both are exceptionally powerful. As employee- 
owner-managers, it is our job to mix and match motivators to the best benefit of the organization; 
consequently, to the best benefit of the employees-owners-managers themselves. 

Direct monetary reward motivators are great for altering direct kinds of behavior, such as 
output, safety awareness, sales, etc., but emotional or feeling motivators are equally important in 
an overall operation. They can be just as powerful for altering direct behavior; I eve they are better 
suited to the more complex and sophisticated side of behavior. Acknowledgment, appreciation and 
praise are all part of a person’s feeling of ownership in an organization. 

Caution must be exercised not to misuse or over-use motivators. They are powerful tools but 
can be dangerous. My worry is not in getting people to work hard, but that they will work too 
hard to the detriment of their health, welfare and safety. We as the owners, have a responsibility to 
ourselves and our respective families. 
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III-A? Beliefs, Principles and Values (0 or 1 point for each item) 

1 . What is the main idea of this passage? 

(The beliefs, principles, and values that should guide this company. 



2 . What should be a major part of this company’s hiring policy? 

(To promote personal growth.) 

3 . Identify the principles that are important to this company? 

_ ... (Tolerance, personal growth, individuality, and motivation.) 

4 . What would be an example of a personal motivator? 

(Accept examples that reflect acknowledgment, appreciation, or praise) 

5 . How are material and emotional motivators the same? 

(Both are powerful. They both change behavior. They both are 
powerful and can be dangerous.) 



Total Score. 
Time 
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_. UNDERSTANDING WHAT YOU READ III-B 

Purpose: The purpose of this activity is to help you to find out how well you understand 

what you read. 

• -----v i rfectitS • re.aff Af teirWitirds, I will'ask you five “questions. You will be able 

to look back if you wish. 

* ... - - - Have students read the selection orally or silently. 

- After reading, ask the comprehension questions below. 

• Score each answer as correct ( 1 points), partial ( 1/2 points), or incorrect (0 
points). 

. ■ - * - - - Score for gist, not word accuracy of response. 

- Allow students to refer back to the passage to answer questions. 

- Record the time required for completing the passage. 

- Record total score 



COLOR PRINTERS FOR EVERY BUDGET 
The Macworld Lab tests 16 ink-jet and thermal-wax printers 

Still procrastinating over making the leap to color printing? You may be running out of 
..excuses; printer manufacturers keep discovering better, faster ways to apply pigments and render 
images, for less money. ... 

The latest color machines still aren’t about to replace monochrome printers - regardless of how 
it works, a color printer is slower, more expensive, more complex, and more finicky about paper 
than its monochrome equivalent. But for business presentation materials, for page proofs, and for 
any application beyond gray-scale, this is a good time to follow the rainbow. 

Or more accurately, rainbows : each of the several color-printing technologies offers its own 
..strengths and financial requirements. Liquid ink-jet printers span the price spectrum, ranging from 
$379 to over $10,000 (see “Finding the Best Business Color”). Half the printers, however, are 
low-end machines priced at well under $1000. These are best for simple applications such as 
printing color presentations and jazzing up reports with colored graphs and spot color. 

Solid ink-jet machines, priced between $5595 and $9995, are popular among publishers, 
thanks largely to an ability to print on any kind of paper. Thermal-wax printers have become the 
mainstays for business color printing, generally offering better output quality and faster 
performance than ink-jets can. Prices for thermal-wax machines have fallen into the monochrome 
laser territory, ranging from $995 to $8999. 

Among the low-end ink-jets, the Apple Color StyleWriter Pro delivers superior image quality 
(though inferior text quality) and is more compact. The HP DeskWriter 560C’s superior text 
quality and networking features give it an edge over the Apple unit but by a small margin. If you 
can live with the Color StyleWriter Pro’s limited networking ability, and image quality is most 
important to you, pick the Apple unit. 

In the solid-ink world, Tektronix’s Phaser 3001 wins hands down. The Data-products Jot PSe 
costs far less, but it can’t print tabloid-size pages and its output is on the fragile side. If you need 
color business documents, consider a plain-paper capable thermal-wax machine instead. 

Tektronix gets the gold in the thermal-wax category, too. Its Phaser 2201 is fast, and its 600- 
by-300dpi resolution yields the best thermal- wax quality output I’ve seen. If your budget barks at 
the $5995 price, consider the 3995 Phaser 220e, which delivers similar quality. 
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' Ili-B. Color Printers for Every Budget (0 or 1 point for each item) 

1 . Identify four problems you would face if you bought a color printer to 
replace your monochrome printer 

- complicated, and require' special 

paper.) 

.2. Identify the three main types of color printers 

(Liquid ink-jet printers, solid ink-jet printers, and thermal-wax printers) 

- . 3 . . . Why are thermal-wax printers often used for business? 

- (They produce better quality and faster output.) 

4 . If you don’t have a lot of money and are concerned about text quality 
and color printing, which machine should you buy? 

(The HP DeskWriter 560C) 

5 . If you have need to print tabloid color pages, which machine would be a 
good choice? 

(Tektronix’s Phaser 300i) 



Total Score 
Time 



Total Score for Part III 
Total Time for Part III 
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Changes in Job Performance: Supervisor's/Co-worker’s Response 

Form 

B.E.S.T. 

program results in changes in the 
workplace. The information in this forth will only be used to evaluate the BEST 
" program. It will not be used by your company to evaluate employees. 



Directions: The participant should complete the information in Part I below and return the form 
to the BEST instructor. The supervisor/co-worker who has been identified should 
~ then complete the rest of the information and return it to 



Part I: Participant information 

Employee/participant: Date 

Company: 

Plant: Shift: 

The name of the supervisor/co-worker who knows my work best is: 



PLEASE CUT OFF THE BOTTOM HALF OF THIS 

PAGE 
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Part Hf Employee Changes 

v: r - --- - . r .. ' Directions: ~ This employee has participated in our program from to 

. Please check die box for each item that best describes the changes in this 

employee since he/she began participating in our program. 

Changes since beginning Amount of Change 

- ’crpaFticipation-rmiih^ - ~ - ----- — 



continues 



greatly 

decreased 


decreased 


no 

change 


increased 


greatly 

Increased 


don't 

know 


to be 
excellent 


does nc 
apply 


1 . This person’s ability to read job-relatd3 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


material. 
















2. This person's ability to write job-relat£l 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


. material"" 
















3. This person's ability to use math to sdU^e 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


job-related problems. 

4. This person’s ability to communicate □ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


effectively at work. 

5. This person’s self-confidence. □ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


6. This person's ability to be responsiblQor 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


their own learning at work. 

7. This person’s ability to work effectivelpwith 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


a team. 
















-• 8. This, person's initiative at work. Q - - 


□ 


□ 


□ 


□ 


□ 


□ 


□ . 


9. This person's knowledge about the jEfc 


□ - 


□ 


□ 


□ 


□ 


□ 


□ 


he/she does. 
















10. The quality of this person's work. □ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


1 1 . The quantity of this person's work. □ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


1 2. The responsibility this person assumSJ in 
— their work. 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


1 3. This person's ability to solve problemEht 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


work. 
















14. This person’s interest in continuing tlaiir 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


education. 
















1 5. This person's willingness to accept □ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


changes at work. 

16. This person’s being respectful of divSfeity 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


and other points of view in the workplace 
















1 7. Involvement in activities designed foO 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


continuous quality improvement. 
















Other (please describe) 

18. □ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


19. □ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


May we share this information with the participant? 


Yes 




No 









Feel free to add comments on the back of this page, if necessary. 
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Pre-Participation Writing Sample - B.E.S.T. 

Name: Date: 

Company: __ Course 

Purpose: This is a chance to find out about your writing ability. The information in this form 

will only be used in the BEST program. It will not be used by your company to 
evaluate employees. 

r : —j -- - --- - -Directions: You are moving up to a new position. To help the person who will take over your 

old position, you have decided to write a description of the responsibilities in your 
former job. Please write this description. You may use the back of these pages for 
notes, etc. 




( 
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- — - v Math Inventory - B.E.S.T. 

Name: Date:_ 

Company: ..Course 



■ Purpose: This inventory will help us plan work for you. If you haven't had to use math over 

the years, you 11 find that you have forgotten a lot. You're not expected to know 
how to do everything, so don t worry if you have trouble. The information in this 
foim will only be used in the BES T program. It will not be used bv vour company 
- - — to evaluate employees. 

Directions: Do as many of these problems as you can. Skip everything you can't do. 

Please do your work on this paper. 



1. 2001 

+1999 


2. 8006 
-2868 


3. 80.9 x .57 = 


4. 32)19616 


5. 8 1/8 +2 1/2= 




6a. 3 2 = 


6b. /49 = 


7 . one fourth = 


-• 


or 





fraction decimal 



8 . Robin is driving to Boston, a trip of 195 miles. She drives a steady 65 miles per hour. If she 
leaves at 1 PM and doesn't make any stops, what time would she arrive in Boston? 



9 . A cleanser is to be mixed with water in a ratio of 1 part cleanser to 4 parts water. If I am 

going to use a gallon of water, how much cleanser do I need? 

10 . 125 centimeters = meters. 

11 . Arrange the following numbers in order from smallest to largest. 

•8 .04 4 0 1.3 1/3 1/4 .0015 
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BONUS QUESTIONS 



12 . 



A certain part, to be acceptable must be within .78 millimeters of its target measurement 
which is 8.10 millimeters. Circle the measurements which are acceptable (i.e. within 



8,95 mm 7.63 mm 8 mm 7.10 mm 8.56 mm 

1 3 . The $450 you saved last year accumulated 7% straight interest. How much did the $450 earn? 



1 4 . In the sketch what is the distance marked with a "?"? 



15. If 3x - 2 = 10, what is x? 



er|c 
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1 -i. Construction Math Inventory - B.E.S.T. 

Name: Date: 

Company: Course, 



Purpose: . This inventory will help us plan work for you. If you haven't had to use math over 

the years, you'll find that you have forgotten a lot. You're not expected to know 
how to do everything, so don't worry if you have trouble. The information in this 
form will only be used in the BEST program. It will not be used bv vour company 
- - - - — to evaluate employees. 

Directions: Do as many of these problems as you can. Skip everything you can't do. 

Please do your work on this paper. YOU MAY USE A CALCULATOR 



1 . Write the following measurements as decimal fractions of a foot (to the nearest 100th). 

3' 6" 2' 3" 5' 1" 4' 4" 

2. Write the following measurements in feet and inches. 

2.5' 4.125' 20.2' 6.33' 

3 . Give the measurement for the question mark in each right triangle. 



4. What is the area of a triangle with a base of 5" and a height of 8"? 



more --> 



Page 43 



" : - 5. If angle..b is. 13.0 and lines A & B are parallel, give the measures of the following angles. 

a= e= j= k= 




6. The sides of the big triangle measure as follows: A = 8', B = 10', C = 14'. If you know that 
side D of the little triangle measures 6' and is parallel to side A, can you calculate the length of 
sides E? If yes, what is the length of E? 




more — > 




63 



BONUS QUESTIONS . 

7 & 8. Give the measures of sides A and C of the triangle. Angle a is 35°. Round your answers 
to the nearest hundredth (i.e. 2 decimal places). 

You must use the trigonometry functioris on a calculator or table. 'If you don't know how to use 
trig, skip this question. 

side A =_ side C = 




9. Figure the yardage of concrete needed to fill the form for the equipment pad shown below. 
It is rectangular and measures 18 feet by 6 feet by 9 inches. 



10. How many degrees is the interior angle (angle a) of an octagonal (8 sided) 
gazebo? 



See drawing below. 
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-Interpersonal Communications and Problem Solving 

B.E.S.T. 



Date: 

Course : 

Purpose: This is a chance to find out how you communicate and solve problems. The 

information in this form will only be used in the BEST program. It will not be used 
by your company to evaluate employees. 

'•'Directions: Please answer the questions that follow. 



Name: 



Company:. 



Part I. Please answer the following questions by circling the answer of your 
choice. ^ 



1 . In a meeting I should not 
let a speaker know that I don't 
understand because it would 



not be polite. 


Disagree 


Not Sure 


Agree 


2. When a group is meeting to 
work on a task, it is important 
to not have disagreements 
because it just creates hard 
feelings. 


Disagree 


Not Sure 


Agree 


3. When there is a problem to be 
solved in a group, it is important 
to find a solution as soon as 
possible. 


Disagree 


Not Sure 


Agree 


4. When there are disagreements 
in a group, there will always 
be a winner and a loser in the 
end. 


Disagree 


Not Sure 


Agree 


5 . The best way for a group to 
make any decision is by a 
vote and the majority wins. 


Disagree 


Not Sure 


Agree 
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: Part H. _ . Rate yourself on a scale of 0-10 with 0 being Never and 10 
being Always on the following questions. Circle the 
appropriate number. 

6. A co-worker doesn’t clean up the work station at the end of the shift as expected. When you 
s’* a thoughtless slob. I would hate to live in your house. 

How could you leave such a mess for me?" 

Never Sometimes Alwavs 

0123456789 1() 



7. Your friend is having trouble with another acquaintance. After listening to a brief complaint, 
you say, "I know exactly what to do. Just tell him to stay off your back. Let him know you 
are not an easy target for getting pushed around." 

Never Sometimes Alwavs 

0 1 2 3456 78 9 10^ 



8. A neighbor is playing very loud music and you are not able to concentrate on your work. You 
say, "When you play your music so loud, I feel frustrated because I am not able to concentrate 
on my work." 

Never Sometimes . Always 

0 1 2.3 .... 4 5 6. 7 8 9 10 



9. You meet a group of three or four friends for dinner. You would like to go to a quiet place 
because you are very tired. One friend insists that the group go to a loud, busy restaurant. 
You say, "I guess it is okay with me." 

Never Sometimes Alwavs 

0123456789 H> 



10. Your family is going on vacation. You want to save time, so you gather information and make 
decisions about where to go and what to do. When some family members object, you say, "I 
know what plans are best for us . All you have to do is just get in the car and come along." 

Never Sometimes Always 

0123456789 10 



1 1 . A friend calls you on the phone and wants to discuss his recent vacation. You have a lot of 
work you need to do. You say, "This is really not a good time for me to talk because I have a 
lot of things I need to get to. Could you call me tomorrow?" 

Never Sometimes Always 

012345678 9 10 



1 2. 1 am aware of the body language of the speaker when I am the listener in the communication 
process 
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Never Sometimes Always 

0 "" 1 2 3 4 5 6 7 8 9 10 



--v v - - - - --y-i-'i-ST : i- ■sp^akr'-tt^for\my ^lf 'ger-itiy-hfee^'^et ' 

Never - . - Sometimes Always 

• o 1 1 2 ; 3 4 5 6 1 8 9 10 



J4. I maintain eye contact with the listener when I am speaking. 

Never Sometimes Always 

0123456789 10 



1 5. When I am listening to someone I let the speaker know how I think he/she is feeling. 

Never Sometimes Always 

0123456789 10 



16. When I attend meetings with a lot of people I don't know well I speak up and ask questions: 



Never Sometimes . Always 

0123456789 10 



17. In situations where I am communicating with people whom I perceive as more powerful than I 
am, I am reluctant to express myself. 



Never 






Sometimes 






Always 


0 1 


2 


3 


4 5 6 


7 


8 


9 10 



1 8. At lunch time you discover that one of your team members would like to ask you for help with 
a problem she is having on her machine but is very reluctant to do so. Is this an example of: 

passive behavior assertiveness aggressiveness 



1 9. Situation: 

The person on the previous shift never cleans the work area. You arrange a time to meet the 
person off shift. How would you tell this person to clean up the work area? 
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2Qv You areJn the middle of a meeting. The item on the floor has caused a lot of discussion and 
confusion with several people talldng at the same time, and nobody seems to know what is 
going on. What can be done by the leader/chairperson to help the meeting run more smoothly? 
Check all the correct possibilities. 



r v -v -- ^^oum^elririeetirig V ' " - 




c. 



d. 



e. 



f. 



Call the meeting to order 

Recess the meeting until everyone stops talking 

Take a break 

Review the agenda 

Make committee assignments 



2 1 . Your daughter is assigned to do dishes on Tuesday and Thursday night. It is Thursday night, 
8:00, dinner was at 6:00 PM, the dishes are still not done. You ask to talk to her. 

Parent: When you don't do the dishes when you are suppose to I get annoyed 

because I can't finish cleaning the kitchen. 

Daughter: Did you have to do dishes when you were growing up? 

Pick the appropriate response. 

Parent: Of course I did, and I had to empty the trash and do the laundry 

everyday before going to school. 

Parent: That's not what I said. 

Parent: That's not the point. I know you don't want to do the dishes, but 

when you don't do the dishes when you are suppose to I get 
annoyed because I can't finish cleaning the kitchen. 

Parent: If you don't finish the dishes by 8:30 there will be no television for a 

month. 
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22. As a member of the employee recreation committee, you are helping to plan a company family 
night. You are head of the entertainment subcommittee. At the next meeting the committee is 
to decide on the entertainment. What would you do to prepare for the meeting? Check as 
many possibilities as you need to. 

... ... . . - "' &.-= Bring cookies for everyone on the committee :: 

b. Investigate local bands, and have a list of the three that you think offer the best deal 

^ c. Survey employees to see what they want for entertainment 

d. Advertise a talent show 

- ‘ e. Find out how much it would cost to have bingo for three hours 

f. Hire a scary clown to entertain the kids. 
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Learning Goals Inventory - B.E.S.T. 



Name: 



Date: 



Company:, 

Purpose: 



Directions: 



Course: 



This is a chance to find out the goals you have for this experience in the B.E.S.T. 
program: The information in this form will only be used in the BEST program. It 
will not be used by your company to evaluate employees. 

Participants may choose to complete this in writing or orally. Please describe the 
goals you have for this experience with B.E.S.T. List them in order from most 
important to next important, and so on. 



1 . 

2 . 

3 . 

4 . 

5 . 



POST 

Purpose: This is a chance to find out how you feel about your progress on the goals you 

identified earlier. The information in this form will only be used in the BEST 
program. It will not be used by your company to evaluate employees. 

Directions: Participants may choose to complete this in writing or orally. Please describe your 
progress on each of the goals you identified earlier. Also, tell about any new goals 
you have now and the progress you are making on them. Check the appropriate 
box. 





None 


A Little 


Some 


Good 


Great 


No longer 














applies 


Goal 1 above. 


□ 


□ 


□ 


□ 


□ 


□ 


Goal 2 above. 


□ 


□ 


□ 


□ 


□ 


□ 


Goal 3 above. 


□ 


□ 


□ 


□ 


□ 


□ 


Goal 4 above. 


□ 


□ 


□ 


□ 


□ 


□ 


Goal 5 above. 


□ 


□ 


□ 


□ 


□ 


□ 


New Goals fiist in order of imDortance to voui 














None 


A Little 


Some 


Good 


Great 




1. 


□ 


□ 


□ 


□ 


□ 




2. 


□ 


□ 


□ 


□ 


□ 




3. 


Q 


□ 


□ 


□ 


□ 





O 
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Process Decisions Regarding the Use of Various Evaluation 

Instruments 



1*. We have two purposes in this project that sometimes conflict: gathering sufficient data to 

demonstrate the effects of participation in the B.E.S.T. program and making participants feel 

welcome as we attempt to meet their learning needs, especially at the beginning of their 
participation. We need to balance these purposes as best we can, always being sensitive to 
participants and their concerns. 

2 . Data on individuals gathered in this project will not be shared with business partners. They 

will only be used by teachers and participants to improve the learning experience. Data on 
individuals will also be sent to the outside evaluator in order to complete the required evaluation 
reports. He will guarantee its confidentiality. 

3 . Aggregate data and all evaluation reports based on these data will be provided to business 
partners apd to the B.E.S.T. project director. These reports will never contain the names of 
any participants. Evaluation reports will also be available to participants. 

4. Instruments developed for this project should be included in an individual portfolio for each 

, _ . . participant along with other sources of information about the learning experience as determined 

by both participants and teachers. These should be kept confidential and reviewed together by 
participants and teachers on a periodic basis. This portfolio should be used as both a learning 
and an evaluation tool. 



Formative Data 

Course Evaluation Form (page 81 

This will be completed by each participant at the last session of each workshop/discreet course 
and at the end of each 10-week brush-up cycle. Teachers will complete a summary sheet for the 
business partners, the employee council, and the project director. The summary sheet will also be 
sent to the external evaluator. 



Summative Data 

Reading Strategies Inventory (page 9 ) 

This will be completed by participants who work on reading. It will be completed during the 
second class session so as to not overwhelm participants who will complete the Reading Inventory 
during the first class session. The post-test will be completed at the final class session or at the end 
of each six-month project reporting period (for participants continuing on into the next cycle.) 

Writing Strategies Inventory (page lOi 

This will be completed by participants who work on writing. It will be completed during the 
second class session so as to not overwhelm participants who will complete the Writing Inventory 
during the first class session. The post-test will be completed at the final class session or at the end 
of each six-month project reporting period (for participants continuing on into the next cycle.) 
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_ ... . ... . Math Strategies. Inventory (page 1 1) 

" ri ' This will be completed by participants who work on math. It will be completed during the 
second class session so as to not overwhelm participants who will complete die Math Inventory 
during the first class session. The post-test will be completed at the final class session or at the end 
of each six-month project reporting period (for participants continuing on into the next cycle.) 

Communications and Problem Solving Strategies Inventory (page 121 

This will be completed by participants who work on communications and problem solving. It 
; will be completed during the first and last class sessions. 

Participant Self-evaluation Form (page 13) 

- ... . . _ This wiU be completed once a year by all participants or when the participant exits (without 

: - planning to return) after completing at least 10 hours of instruction. 

Teacher’s Perception of Improvement Form (page 16~) 

This will be completed once a year by teachers for all participants or when the participant exits 
(without planning to return) after completing at least 10 hours of instruction. 

Reading Inventory 

This will be completed by participants who work on reading. It will be completed during the 
first class session. The post-test will be completed at the final class session (after ten hours or more 
of instruction in reading) or at the end of each six-month project reporting period (for participants 
continuing on into the next cycle.) 

Changes' in Job Performance: Supervisor’s/Co-worker’s Response Form 

This will be completed once a year by supervisors/co- workers for all participants or when the 
participant exits (without planning to return) after completing at least 10 hours of instruction. 

Pre- and Post-Participation Writing Sample 

This will be completed by participants who work on writing. It will be completed during the 
first class session. The post-test will be completed at the final class session (after ten hours or more 
of instruction in writing) or at the end of each six-month project reporting period (for participants 
continuing on into the next cycle.) 

Math Inventory 

This will be completed by participants who work on math. It will be completed during the first 
class session. The post-test will be completed at the final class session (after ten hours or more of 
instruction in math) or at the end of each six-month project reporting period (for participants 
continuing on into the next cycle.) 

Interpersonal Communication and Problem Solving 

This will be completed by participants who work on communications and problem solving. It 
will be completed during the first and last class sessions. 

Learning Goals Inventory 

This will be completed by all participants during the first and last class sessions. The post form 
will be completed at the final class session (after ten hours or more of instruction) or at the end of 
each six-month project reporting period (for participants continuing on into the next cycle.) 



O 
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BEST PROGRAM EVALUATION 



Interviews conducted with employees at 
General Electric Corporation, John A. Russell Corporation, 
and th© Rutland Regional Medical Center 



by 

Robert T. McLaughlin, Ph.D. 
Director 

Vermont Workplace Education Program 



May 1995 
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BEST Evaluation 



Goal The aim of this evaluation effort was to ascertain the relative effectiveness of 

aei^ces ^ John A. Russell Corporation, and 

the Rutland Regional Medical Center. 

Methods The assessment was performed by interviewing BEST instructors as 
well as individuals within these three firms to determine: 

r . A. ~By what criteria do stakeholders with diverse perspectives assess 

whether a workplace education service is “effective”? That is, what is 
each stakeholder’s operational definition of effectiveness? 

B. Applying these criteria, to what extent do the stakeholders regard the 
BEST Program’s services as having been effective? 

Interviewees included program participants, supervisors, and other company 
officials with important perspectives on employee education and performance (e.g., 
education and human resource directors). The questions were posed open-endedly, 

' using standard qualitative research methods. That is, questions were posed in a 
manner to ensure that respondents’ own ways of thi nkin g about the issue of 
“program effectiveness” were elicited rather than asking whether respondents 
agreed or disagreed with the validity of using effectiveness criteria posed by the 
interviewer. Two basic questions were posed in each interview (see below) - one to 
elicit respondents’ definitions of program effectiveness, the other to ask them to 
apply their definition to give an assessment of the extent to which they found the 
BEST Program to have been effective up to this point. 

In each interview, once respondents had indicated they had exhausted their ideas, 
the interviewer summarized the points raised by the respondent, to be sure he had 
heard and understood the respondent correctly. The next step involved presenting 
the respondents with a draft of the evaluation report for their critique - the 
qualitative research step known as a “member check” to ensure that the report 
provided an accurate representation of the respondents’ insights and ideas. This 
review process only elicited comments from two stakeholders, both staffmembers of 
the BEST program, but their comments underscored critical issues not reflected by 
respondents initial statements as well as highlighted the limitations of relying 
solely on interviews to gauge program effectiveness. At the same time, an 
important goal of this evaluation was to identify criteria which diverse stakeholders 
regard as important criteria by which to assess workplace literacy/education 
program effectiveness. This knowledge can and should guide subsequent data 
collection using a mix of research methodologies (observation, student assessment 
results, etc.). The following presents a compilation of individual interview 
comments. 



Results 



whether a workplace. education effort 

at (your institution) is effective 1 ? 

General Electric 

A. Students/Employees (seven interviewees) 

1. Become confident using computers. 

2. Become confident helping others use computers. 

3. Gain new knowledge. 

4. Increase self-esteem. 

5. Feel comfortable being in a classroom. 

6. Feel comfortable admitting do not know something and asking for help. 

7. Become confident when have to learn something new. 

8. Program protects confidentiality of students. 

9. Supervisors supportive of employee participation. 

"10. Manner in which teacher interacts with students -- respect, caring, patience. 

11. More employee participation in group efforts (e.g., committees, team 
discussions, etc.) due to increased confidence and self-esteem. 

12. Reduce absenteeism as employees feel more valued and morale increases. 

13. Increase effectiveness of future training efforts as employees will be more 
confident about learning. 



B. Human Resources Department Leader (one interviewee) 

1. Reduce manufacturing losses and scrap. 

2. Improve quality of output. 

3. Strengthen employees’ math and literacy skills. 

4. Increase level of employee participation in education. 

5. Increase employee self-esteem. 

6. Reduce rate of loss of longer-term employees (e.g., those that have been with the 
company for 15 years or more) because they cannot cope with the rapid rate of 
change in this workplace. 

7. Improve the atmosphere of the plant in terms of attitudes towards learning and 
education. 

8. Increase the numbers of employees who bid on jobs - an indicator of increased 
skills and/or confidence to handle new challenges. 

9. Increase rate of promotion from within the company. 

10. Increase employees’ willingness to discuss any skill deficiencies they may have, 



so that solutions can be sought openly. 



C. Education Department Leader (one interviewee) 

1. Increase employee self-confidence. 

2. Increase employee bids for.upgrade positions. 

3. Increase number of employees taking courses elsewhere after getting GED or 
basic skills. 

4. Exercise social/civic responsibility by investing in employees’ education. 

5: Help employees become more outgoing. 

6. Increase number of employees volunteering to be on task forces, employee 
associations, focus groups, continuous review boards, etc. 

7. Increase employees’ literacy, numeracy and other essential skills. 

John A. Russell Corporation 

A. Education Department Leader/Assistant Treasurer (one interviewee) 

1. Provide better and more training for employees. 

‘2. Help, employees have more opportunities for promotion and advancement. 

B. Personnel Department Leader (one interviewee) 

1. Program to be non-threatening to employees - a safe environment for learning. 

2. Confidential, so that no one knows what skill level a person has attained. 

3. "Will help employees do their work more safely. 

4. Will provide employees with skills in the workplace but also at home. 

5. Offerings should be at hours convenient to employees. 



C. Students/Employees (three interviewees) 

1. Program to be non-threatening -- a safe environment for learning. 

2. Provide learners with skills that can use on the job. 

3. Help employees become more confident about being in a learning situation. 

4. Encourage students to learn at their own rate, allowing them time to master the 
course content. 

5. 

Rutland Regional Medical Center 
A. Education Department Leader (one interviewee) 



1. Positive feedback from managers. 



2; Positive feedback from employees. 

3. Participants being better equipped from promotion from within, laterally or 
vertically. 

4. More self-cpnfidence among participants -- attitude changes^ perceptions by 

•“btfi&rs of oni^s'wrKT" - 

5. Moving beyond basic courses to more advanced ones. 

B. Personnel Department (one interviewee) 

-1. Attendance in courses ■ 

2. Written evaluation by course participants 

3. How current and important the course topics are 

4. Informal feedback by participants 

C. Students/Employees (five interviewees) 

1. The teacher’s style and manner 

2. The program’s topics -- their relevance and usefulness. 

3. Help with skills that are important. 

-4. Look for interesting topics, and topics that would help with work and 
employability and promotability. 

5. Encourage participants to pursue further education. 

6. Want a relaxed, safe atmosphere in which to learn. 

7. Want to cover content in an efficient manner. 

— BEST Program Instructors (four interviewees) 

1. Increase employee self-confidence. 

2. Increase employee assertiveness. 

3. Improve employees’ handling of ridicule by others. 

4. Improve employees’ handling of own emotions 

5. Increase employee skills in communicating about personal issues. 

6. Increase employee proficiency in reading, speaking, math and other basic skills. 

7. Expand the number and percentage of employees pursuing educational 
opportunities. 

8. Increase employees’ confidence in undertaking cross-training. 

9. Increase employees’ effectiveness in working as a team. 

10. Engage supervisors and group leaders in consistently supporting subordinates’ 
participation in education. 



1L Applying these criteria, to what extent has the BEST Program been effective ? 
Please explain why. 



General Electric 

A. Students/employees 

1. Have become confident using computers. 

2. Have become confident helping others use computers. 

- 3. Have gained new knowledge - e.g., about basic math, reading, handling 
paperwork, knowing where to find information. 

4. Have increased self-esteem significantly. 

5. Have felt comfortable being in a classroom -- don’t feel foolish asking questions, 
all employees are treated equally and with respect, can get one-to-one tutorial help, 
glad there are no tests. 

6. Have felt comfortable admitting do not know something and asking for help. 

7. Have become confident when have to learn something new. 

8. Program has tried to protect confidentiality of students - but this is still a 
problem. Until the company offers wider range of courses at different skill levels, 

-co-workers will know that being a student means having low basic skills. 

9. Only some supervisors have been supportive of employee participation — this 
varies somewhat from one supervisor to the next and remains somewhat of a 
problem, though this is getting better as more supervisors seem to respect the value 
of the program. 

10. Now are more confident even in non-work settings -- e.g., when having to keep 
a check book balance and filling out doctor’s office forms. 

11. Are experiencing less stress generally because understand more. 

B. Human Resources Department Leader 

1. Have heard some anecdotal indications of improved output from program 
participants. 

2. Anecdotal data suggesting program has strengthened participants’ math and 
literacy skills. 

3. Some signs that BEST has improved the atmosphere of the plant in terms of 
attitudes towards learning and education. 

4. Several supervisors have said they are noticing big improvements in 
participating employees’ self-confidence, assertiveness, comfort in asking questions, 
and confidence in approaching new challenges. 

5. Participants are referring others to the program, an excellent indication that 
these employees are feeling well served. 



irt: -V"-- Education-Department Leader 

1. Anecdotal data suggesting increase self-confidence among participants. 

2. Anecdotal data indicating participating employees have become more outgoing. 

' v - v : Stiidfe^ has led to ah increase 

employees’ hteracy, numeracy and other essential skills (though want firmer data 
- on this important criterion of program effectiveness). 

Member Check Comments: One stakeholder felt that the positive assessment by 

- -students/employees participating in the program at GE was an overly rosy 

assessment colored by respondents’ desires to tell the evaluator “what he wanted to 
hear”. The stakeholder said that this assessment did not reflect employee morale 
issues which, to this person, suggested that management probably did not support 
employee-investment strategies such as BEST as strongly as the interviews 
appeared to suggest. A second stakeholder felt, however, that the interviews of GE 
employees accurately reflected upper management’s strong support for BEST while 
low employee morale was the result, instead, of front-line supervisors who “are 
responding to many stressful and conflicting pressures”. Neither stakeholder’s 
comments appear to invalidate the statements expressed by the initial interviewees 

- - . .. . -who expressed (a) highly positive marks for BEST by program participants and (b) 

strong leadership support at the plant. 

John A. Russell Corporation 

A. Education Department Leader/Assistant Treasurer 

1. Having a problem getting employees to' commit to BEST offerings and would like 
assistance from BEST Program to come up with effective strategies to recruit 
employees into the program. 

2. Aren’t sure what else can do to encourage more employees to sign up - job stress 
is a factor for some as they have little time or energy to do anything more in their 

fives. 

3. The employees who have been faithfully involved see the program as very 
valuable -- providing a respectful, safe learning environment which helps them gain 
important skills. 

4. Might want to survey employees about whether they would be interested in 
signing up for courses/workshops if they could be offered at different times. 

5. Perhaps we (company) have not focussed as much as we should yet on addressing 
this problem of recruitment. 

B. Personnel Department Leader 

1. Not many have signed up because they are at a crucial point in one job site 
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: - wherecan’t afford to take work time and because, more generally, the work force is 

stretched to the limit with the volume of work that must be done. 

2. Employees who have signed up have said they don’t feel threatened, embarrassed 
or stupid. They have indicated the courses are a very positive and useful 

3. Hadn’t realized until this week that not many employees have participated so far 
in the BEST offerings. (The evaluation interview seemed to be a helpful catalyst to 
reflect on extent to which employees have used the program and found it useful.) 

- -_j-.-J.-_.-,. C; Students/Employees 

1. Have found the courses to be very useful. 

2. Have liked the way the courses were arranged, the personal help, the supportive 
manner of the program personnel, and the completely positive and non-threatening 
environment created by the instructors. 

3. Have felt frustrated only because of the pressure of so much work making it 
impossible to attend classes sometimes. 

4. Has helped employees with self-esteem. 

5. Instructors have been very pleasant and encouraging. 

_6. The topics have been-important for employees’ future growth and careers. A 

wide variety of topics have been offered -- more than have time to take. 

7. At start of the first course taken, was a surprise (one employee commented) 
when asked to articulate what one most wanted to learn (rather than be told by the 
teacher what would learn in the course). An orientation about this would be helpful 
at the very start of a course, for those not used to learning in this self-directed 
manner. 

8. Like being able to learn at own pace, emphasizing mastery rather than “covering 
the material”. 

9. Wish more employees would be willing to sign up -- many are reluctant to admit 
they have trouble reading and in exercising other basic skills. So, would be good to 
offer private, confidential tutoring to increase recruitment. Another obstacle is 

offering the courses outside of company time. This is a problem because Russell 

employees already are working long hours and, often, in towns distant from 
Rutland on a job site. Therefore, employees are reluctant to stay after hours and to 
travel to a possibly distant classroom to pursue their studies. 

10. (One student said) Wish class sessions would be longer — seems like just when 
you get into a lesson and it’s time to go, then have to pick up where you left off a 
week ago. 

11. Also should have a video to show employees to encourage them to sign up for 
BEST courses. 

12. One student said this was the most beneficial thing he’s done in a long time. 
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>. Member Check Comments: One stakeholder commented, upon reviewing the initial 
summary of interview responses, that BEST program staff and company officials 
may need to “make B.E.S.T. a priority within Russell .” A second stakeholder 
wondered whether respondents’ suggestions to make classes easier to attend by 
: Adjustments would be sufficient to increase 

enrollment: “is management really committed?” this person asked. 

Rutland Regional Medical Center 

. - — A. -Education Department Leader: 

-Several participants feel renewer self-confidence. 

-Several have gone on beyond BEST already -- this is an important measure. 

- Have had promotions of BEST participants already. 

- Have had some failures so far -- haven’t identified well enough what BEST can 
offer and cannot, within the federal guidelines; and the Education Dept. Needs to 
follow-up on technical and more advanced topics desired by employees 

- The assessment process has provided very useful data. 

.■» --- . . -B.~ Personnel Department Leader: 

- Courses and assessment'process very relevant to organizational change process 
underway at RRMC. 

- Many employees still in denial about their own skill deficits. 

- Feedback in self-evaluation and evaluation by supervisors has emphasized need 
for-training in time management, working with difficult people, study skills — topics 

which BEST has tackled through workshops. 

- When BEST came in, it wasn’t marketed as a literacy program but somehow a 
stigma did become attached to the program. So, RRMC has worked hard to make 
BEST more than a program for very basic skills. 

- Informally, the feedback has been very positive -- employees have felt it’s been 

fortunate to have such a program offered to them. 

C. Students/ Employees 

1. The teacher’s style and manner have been excellent. 

2. The program’s topics have been relevance and useful. 

3. The program has helped with skills that are important -- e.g., writing. 

4. Many people have taken advantage of many offerings. 

5. Some have continued on to higher learning as a result. 

6. Classes have maintained a safe, comfortable atmosphere. 

7. Classes have covered a lot of content in a short time. 

8. Some are looking for topics that BEST can’t offer and the Education Department 
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is now working hard to collaborate with BEST personnel in designing these 
additional, highly desired topics. 

9. BEST has helped spur a lot of people to consider education who may not have 
thought about it for years. 



Member Check Comments: One stakeholder felt that the “cooperation-experienced 
between the education department, human resources and [the BEST program] is 
not reflected in the evaluation... .There is a large amount of support at the hospital 
for BEST. But... finding the match between employee interest and scheduling 
classes at opportune times appears to be elusive.” This individual recommended 
tracking attendance patterns over time in order to determine the best scheduling 
strategies to conform to fluctuating demands on employees. Another stakeholder 
commented that the efforts of BEST program staff to align the program with the 
organization’s goals and education and training needs “has created an authentic 
and productive partnership” between the RRMC and BEST, “even if attendance is 
elusive”. 



BEST Instructors 

1. Have increased participants’ self-confidence, assertiveness, handling of ridicule 
by others, handling of own emotions, and skills in communicating about personal 
issues. 

2. Have increased participants’ proficiency in reading, speaking, math and other 
basic skills. 



Conclusions 

1. Instructional Quality: There was unanimous agreement among respondents 
that the quality of instruction delivered by BEST personnel has been high. 

2. Logistics: Respondents at the John A. Russell Corporation indicated some 
logistical concerns and suggested improvements, while respondents at all 
three workplaces cited favorably the willingness of BEST personnel to adapt 
to each workplace’s unique logistical constraints. At Rutland Regional 
Medical Center, respondents noted the willingness of students and 
instructors alike to deal with the necessary obstacle of erratic attendance due 
to the inherently unpredictable demands of work in a hospital. At General 
Electric, respondents acknowledged grateful the instructors’ willingness to 
offer instruction at all hours of the day and night, to accommodate to a three- 
shift operation. At Russell, meanwhile, respondents suggested that the 
possibility of offering courses and workshops are hours better suited to 
employees’ current work schedule should be explored. 



Employee recruitment: At all three workplaces, respondents noted concerns 
about recruiting more students into the program. Because respondents 
unanimously regarded the program as “highly valuable”, they said they 
wanted more employees to benefit from the program. Several respondents in 
'■ wdri^laces said that some" stigma is associated with participation 

in the program because it is seen as serving those with the weakest skills. 

At the hospital, several respondents said that some success has been 
achieved recently in counteracting the stigma; at General Electric and, 
especially, at Russell this remains a serious problem, according to 
respondents. 



Recommendations 

Continue offering courses on company time, as this is valued and appreciated 
highly by employees. 

Consider revising tuition reimbursement policy at General Electric so that 
company will still pay even if employee fails a course (so long as attendance 
record in course is perfect or near-perfect -- i.e., so long as employee attends 
and tries hard to succeed). I recommend strongly that this option be 
explored because several employees noted during separate interviews at 
General Electric that they and most co-worker are reluctant to take 
advantage of the company’s tuition reimbursement benefit because they fear 
they might fail. 

Given the plant leadership’s clear desire that more employees take advantage 
of this employee benefit, it would be very worthwhile to conduct an 
experiment of limited duration — i.e., six to twelve months — during which 
employees could receive full tuition reimbursement for courses even if they 
failed until they have failed a second or, perhaps, even a third course. The 
company’s understandable concern that some might employees might abuse 
this benefit by enrolling for a course and not investing themselves in it 
sufficiently to pass it is, I believe, discouraging an undetermined number of 
employees from seeking employee-paid educational offerings because they 
lack confidence in themselves -- not because they would be likely to abuse the 
company’s generosity. 

One useful step in researching this option would be to survey the work force 
and/or interview a random representative sample of employees to ask them 
whether they would be more likely to sign up for courses if they knew both 
that (a) the company would insist on a high rate of attendance (and/or other 
proxy indicators of student effort) yet (b) the company would still provide full 
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... reimbursement were the employee to fail the course (up to a maximum 
number of failed courses to be specified by the company). - 

3. Develop a data gathering system to validate the extent to which employees 

^ Utilize pre- 

tests and post-tests to measure gains for each participant, based on clear 

. benchmarks/desired outcomes. Representatives from the three organizations 
should meet with BEST personnel to devise a common approach to tracking 
participating employees’ learning gains. A student portfolio system could f 
- _ . . ... easily he used, whereby each employee maintains a portfolio of products 

- ~ " demonstrating extent of mastery of key skills and knowledge over time, at 

intervals. Just such a meeting has been set for May 26th to determine how 
best to assess and report on student learning gains. 

4. Encourage each company’s employee education committee to ask 
participating employees to tell committee members of their assessment of 
courses and workshops. Participating students should be encouraged 
especially to notify committee members of any concern or problem they have 
with a course or workshop, so that the committee can discuss the issue and 
notify the BEST Program and appropriate company officials of a perceived 

concern. 

5 In General Electric and RRMC, some employees noted a concern that the 

courses and workshops were not being clearly advertised as targeted to those 
needing to strengthen basic skills; this has resulted in some participants 
feeling that educational offerings were too basic for them. This concerns 
appears to apply more commonly to the workshops than to the courses. 

6. On the other hand, interviewees noted in every setting that the relatively 
narrow range of offerings available through BEST (due to federal funding 
guidelines and company resource limitations) has led employees generally to 
regard BEST offerings as for the least educated -- resulting in some degree of 

stigma. 

7 To reduce or eliminate the stigma associated with BEST offerings, it would, 
of course, be very helpful to be able to offer a wider range of courses and 
workshops of interest to employees at all skill levels. Given the impossibility 
of using federal funds for other than basic skill topics and the unavailability 
■ of corporate resources, it is strongly urged that the employee education 

committees work closely with their respective top managers to enlist 
; employees knowledgeable in a wide range of desired topics (as highlighted in 

the assessment surveys) to volunteer to teach courses and workshops geared 
for a wide range of skill levels, related to the most popular topics. BEST 
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Program personnel and delegates from each employee committee could then 
meet together to explore whether and how to offer volunteer-taught offerings 
so that employees from any of the three organizations could attend. Thus, a 
first aid course taught by an RRMC employee, a hunter safety course by a GE 

a Russell employee, 

could be offered to anyone interested from the three organizations. These 
. offerings should be publicized as BEST Program services, along with the 
federally funded courses and workshops. 

- ... _ Such a joint employee education committee - i.e., drawing representatives 

~ ' from the three employee committees - also could explore valuably how to 

encourage course participants to donate time as child care providers on a 
rotating basis so that employees with young children could afford to 
participate in BEST offerings after their regular work hours. 

8. BEST personnel should provide, as soon as possible, intensive support to 
Russell Corporation officials with regard to instituting effective methods for 
recruiting employees into the program. 

9. Explore with company officials the value and feasibility of instituting simple 
processes by which supervisors of participating employees periodically assess 
the perceived impact of program participation in terms of outcomes for 
participants such as the following: 

* increased effectiveness in handling paperwork, reading, performing 
arithmetic tasks, etc. 

* increased morale, self-confidence, assertiveness, willingness to volunteer, 
comfort in participating actively in group activities, etc. 

* increased willingness to ask for assistance 

* increased output 

* reduced rate of scrap/mistakes/errors 

* increase in safe practices 

* improved relations with supervisor, managers, etc. 

* improved relations with peers 

* increase in bidding for positions 

* increase in confidence when faced with having to learn new skills 

* increased effectiveness in handling stressful situations 

One strategy that should be explored is to ask supervisors to complete a 
questionnaire rating their subordinate relative to the above dimensions when 
the employee first starts taking a BEST course, then during the course, and, 
finally, after completing the course. This data could be provided to the 
employee education committee, top management, and BEST Program 
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;• personnel on a- regular basis. I would be pleased to draft a simple instrument 
for this purpose, should the companies and BEST personnel desire it. 

10. Consider providing an orientation at the start of each BEST course, 
explainirrg to students that they may-ejqDerience some confusion at first about 
the way in which learning will proceed in a self-paced, self-directed manner -- 
i.e;> in a way very unfamiliar to most adult learners. While all interview 
respondents expressed a strongly favorable view of the manner in which 
BEST offers instruction, one employee suggested that this approach is 

- ... . . ._• initially: disorienting and made him feel uncertain. It might be useful to ask 

students currently enrolled in BEST courses whether they shared that 
experience at the start of their enrollment. 

11. Bring together company and program officials to explore possible strategies 
for increasing front-line supervisors understanding of and appreciation and 
support for their employees’ involvement in the program. 
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